Observation : A shifted transversal design (STD) is similar to the adjacency matrix of an (unbalanced) expander graph.
STD(n;q;k) is a matrix construction algorithm that accepts parameters : n,q and k to produce a (qk x n) 0-1 matrix with certain special properties. Each of the n columns has k 1’s in it, which are distributed, one in every q x n block. Each row has usually 1’s in it. Such matrices are very useful as pooling design schemes and error-correcting codes, among other things. The STD algorithm is due to N. Thierry-Mieg.
Expander Graphs are sparse but highly connected graphs. From here, we learn that a bipartite graph H = (X,Y,E) is said to be an ()-expander if |X|=n, |Y|=m, the degree of each node in X is d, and for every
, the set of vertices
that are adjacent to A satisfies
. A higher value of
implies a better expansion property.
The connection between STD’s and EG’s stems from the fact that STD(n;q;k) creates n distinct (actually disjunct) columns of length qk with column weight k and this is similar to the adjacency matrix of an unbalanced expander with left degree k and |X|=n and |Y|=qk.
To construct such a matrix, via STD, we do as follows :
Given the parameters n, q and k, the binary matrix STD(n;q;k) can be constructed as follows. The STD has a layered construction consisting of k layers of q x n binary matrices. For all
, let
be a
boolean matrix representing layer
, with columns
. Let the circular shift operator,
, be defined as,
and
. Note that
is a cyclic function and when applied
times maps
onto itself,
,
. To design a layer
, for all
construct
, where,
if :
if :
where
The layers
are put together to form
by,
.
Note : q is a prime number. To get the required m rows with column weight k, try to find the smallest prime,
and
.
In fact, any constant column-weight d-disjunct matrix has such a property. This property of d-disjunctness seems to be related to linear time decoding capabilities of codes. Not surprisingly, the STD is similar in nature to low density parity check (LDPC) codes.
The point of all this being that, STD has a very fast and clean construction method which could come in handy while creating expander graph-type structures (and codes with error-correcting properties). This is especially useful for the compressed sensing methods that use sparse matrices and the MATLAB codes that go with it.
% A STD-based method to create the sparse measurement matrix (used for compressed sensing)
% Here k is the number of ones per column
% qk is the number of rows
% So if m rows are needed, choose q : prime >= m/k and k<=q+1
% n is the number of columnsfunction M=std(n,q,k)
if rem(log(n),log(q)) == 0
Gamma=log(n)/log(q)-1;
else
Gamma=floor(log(n)/log(q));
endflag=0;
% Check if STD Correction is needed
if k == q+1 && floor((n-1)/q^Gamma) < q-1
flag=1;
endformat short
M=zeros(k*q,n);
s=zeros(k,n);C00=zeros(q,1);
C00(1)=1;for j=1:k
for i=1:n
if j-1 < q
for c=0:Gamma
s(j,i)=s(j,i)+ ((j-1)^c)*floor((i-1)/q^c);
end
elseif j-1==q
s(j,i)=floor((i-1)/q^Gamma);
endM((j-1)*q+1:j*q,i)=circshift(C00,s(j,i));
end
endif flag==1
M((k-1)*q+(floor(n/q^Gamma)+1)+1:k*q,:)=[];
end%End of Function

16 comments
Comments feed for this article
March 6, 2008 at 4:29 am
Igor Carron
Hello Raghunandan,
Shouldn’t the first line of your matlab code states something like
%An STD-based method to create the sparse measurement sensing matrix
instead of recovery ?
Also in my version of matlab I don’t seem to need Return M
Igor.
March 6, 2008 at 8:02 am
raghumk
Hi Igor
Yes, you are correct on both points.
I use STD’s in the context of group testing and observed that in the CS context it maps to the sparse measurement (and recovery) problem of Berinde and Indyk. In fact, it has very good decoding properties arising from its structure and, in my experience, faster construction for any column weight d.
In Matlab it is not necessary to specify a return statement.
Thanks for the corrections.
March 6, 2008 at 3:33 pm
Radu Berinde
Hi,
In other words, s(i,j) tells you the position of the 1 within “slice” j of column i, is this correct?
I am a little confused by the fact that the claim is true “with high probability” but the construction is not randomized. Does that mean that it is true for most choices of parameters n,q,k?
March 6, 2008 at 3:49 pm
raghumk
Hi Radu
You are correct about s(i,j) determining the position of 1 in the layer j, for column i.
About the “with high probability” statement. It was my understanding from your paper that this kind of a matrix produces “the adjacency matrix of an expander graph of degree d with high probability (for a proper value of d)”. Since this deterministic method essentially imitates the properties of the random construction, I put that statement in order to not make a larger claim.
I would be grateful if you could clarify this point for me. Thanks.
March 6, 2008 at 5:09 pm
Radu Berinde
Hi Raghunandan,
The claim is that a random matrix with D 1s on each column is with good probability an expander.
That does not necessarily imply that this particular construction is an expander. In fact, as far as I know, an explicit construction of an unbalanced expander with small right-set size (i.e. m = O(alpha*n*log(n)) in your definition above) is an important open problem.
Of course, even if this construction is not an expander, it might still work for compressed sensing, and even if it does not in theory, it might still work in practice (the STD matrix does look really nice in terms of column incoherency). I will be sure to run some tests and keep you updated. This construction is very interesting because it allows operations (e.g. vector multiplication) without explicitly generating and storing the whole matrix.
Thank you very much for the post!
March 6, 2008 at 5:20 pm
raghumk
Thanks for the clarification. I should probably reword the post.
I have checked that in practice STD’s work very nicely in compressed sensing applications (albeit biological ones). In fact, the LP decoding is much faster with STD matrices than with random ones (because of the column incoherency, probably). I would be excited to see how the CS community takes this up.
March 6, 2008 at 8:19 pm
Radu Berinde
Hi again
I have run some tests and it does not seem to work in the sparse recovery framework, at least not with comparable parameters.
As a “counter-example” which you can easily check: consider N = 200, Q = 10, K = 10. Random matrices with these parameters result in exact recovery of vectors with 20 non-zero components (I ran this experiment hundreds of times for many types/parameters of matrices and it never failed).
Let A be the STD matrix for these parameters (generated with the code above).
The matrix has 200 columns and 100 rows. Consider a signal vector x of size 200 such that x(1), x(2), .. x(20) are all 1, and the rest are all 0:
x = [ones(20,1);zeros(180, 1)];
The result of A * x happens to contain 2s everywhere. Also, the matrix A happens to have exactly 20 1s on each line. Hence another vector x’ which satisfies the system Ax’ = Ax is the vector which has 0.1 everywhere. Both x’ and x have the same l1 norm, which means that x is not LP-recoverable (since |x’|_1 <= |x|_1 but x’ != x).
Note that the same thing also happens when x is only 10-sparse (the first 10 elements are 1, the others 0).
This shows that A is in fact not an unbalanced expander (if it was, the recovery should have always worked).
Best,
Radu
March 8, 2008 at 1:40 am
Radu Berinde
A small update – I ran experiments with the boat image (in the sparse matrices paper), and it sometimes works acceptably but usually it adds some very nasty artifacts to the recovered images (different patterns of darkness). The fewer the measurements, the more serious the problem is (for M around 7500, most of the attempts resulted in an image which is completely destroyed).
This is discouraging as the “good” recovery matrices I’ve tested – sparse and scrambled fourier – are very reliable in recovering the image, and they yield recoveries with about the same quality.
Also, I seriously doubt that this matrix is a good expander, simply because it would be a very significant result which someone would have noticed by now
March 8, 2008 at 8:56 am
Raghu
Hi Radu
Thanks for the follow-up. Its good to know, before one ventures too far.
In the group testing sense, the q and k have to be chosen to “guarantee” the exact decoding of a d-sparse signal with E measurement errors, such that,
As you might have experienced, the
accurately decodes for values of d and E much larger than the “guarantee”.
March 8, 2008 at 12:43 pm
Radu Berinde
On the other hand, I ran many experiments with truly sparse random vectors (K peaks of value +/- 1), and they seem to work just as good as random sparse matrices on these inputs.
There is definitely something non-trivial going on.. It is possible that STDs lack some properties that allow random matrices to recover nice results in the non-sparse case (like images).
March 8, 2008 at 1:15 pm
Raghu
Yes, that is what I have noticed too. As long as the signal is close to being k-sparse it works great (and faster). Power law distributions will do too, but only if the lower terms are fairly small compared to the peaks. I guess the in-built structure helps with speed but costs us accuracy (when operating on non-sparse signals).
March 8, 2008 at 1:59 pm
Radu Berinde
I do not think it is faster than random sparse matrices. It is certainly a lot faster than other dense matrices. The nice thing about it was that it allows you to do matrix operations without explicitly storing the matrix (which is nice when memory is a problem, as well as for streaming algorithms).
March 8, 2008 at 5:21 pm
Igor Carron
Why don’t you guys write something about this ? making it a feature the ability to not have to store the matrix. I am sure people could find a use if they have a good prior knowledge on the signal.
Radu, you used the Min TV for the boat image reconstruction ?
Igor.
March 8, 2008 at 5:45 pm
Radu Berinde
No, I used LP recovery on the wavelet coefficients vector.
March 9, 2008 at 5:34 am
Igor Carron
Great, it may show the need for better decomposition (i.e. compressibility) with more appropriate dictionaries of functions like curvelets / wedgelets / contourlets.
Igor.
August 6, 2009 at 10:13 pm
Anonymous
i cant understand the things that are listed above here. can you kindly elaborate?