DSSS framegen/framesync

[brian-armstrong]

I think it would be interesting for liquid to have support for DSSS frames. This seems like it shouldn't be too hard since the qdetector already aligns timing on an msequence. My limited DSP knowledge suggests that the header and payload could just use the same msequence as the preamble, and then the sync needs to run qdetector and cross-correlation.

I think I have enough to go on that I might try to implement such a thing. Would it be likely to be accepted as a patch, if it was documented/tested?

[jgaeddert]

Sure! I have a branch on this from a long time ago. The only issue with using the qdetector is that in order to get the huge spreading gains generally found with DS/SS, you can't rely on a single sequence for detection, but need multiple. Having said that, the qdetector is probably a good place to start. I was thinking about this a while ago and how I would implement it efficiently. I'll dig up my old DS/SS branch and see what I can find.

[brian-armstrong]

Honestly, I think the spreading gain it has already is perfect for my use case. My room-sized tests showed that the preamble typically picked up, which suggests to me that the msequence that's there is probably pretty well suited for this job. I'm even considering moving down to 5 or 6 bits so that I can also use some pulse shaping and still have some throughput left. Not a lot of room in 20kHz :)

[jgaeddert]

Let me work on this for a little bit. I might have something useful for you.

[brian-armstrong]

I'm making a serious attempt at this, though I'm still interested in seeing your implementation. I figure if nothing else, trying this would help me learn more DSP.

One thing I'm not so sure on is how to handle syncing. I had only read as far as the preamble on flexframegen and assumed syncing happened on that, but now I see more syncing occurs on pilot signals inside the header. It feels to me that a frame made entirely of m-sequences ought to actually be pretty easy to keep syncronized from clock drift, but I'm not really sure if it's necessary.

If I start out just syncing/equalizing a preamble section and then decoding everything that follows, do you think that'd work at least in good conditions? I'm sort of hoping I can start there and then empirically tune from there. It feels like some correlator could be used continuously on the sequences that follow but I'm not quite sure how to go about that, or if it's actually necessary.

Also, do you think it makes sense to use the same sequence for preamble, header and payload, or to vary to shorter sequences after acquisition? I guess more empirical testing is probably the answer here. 2**7-1 symbols per bit is pretty darn slow, though that is the point here anyway.

[jgaeddert]

A few comments:

1. the flexframe uses a long preamble for detection and coarse carrier phase, frequency and timing offset correction
2. the header and data portions use pilots to provide fine carrier phase/frequency correction
3. The detection of a frame should be very robust, but the remainder of the frame should reflect the channel state. That is, there would ideally be feedback to adjust the spreading gain of the payload based on how good or bad the channel is
4. ultimately you might want to equalize, but you can probably start off without that for the moment

[brian-armstrong]

I see, that makes sense. But if you were just sending a stream of m-sequences (or Gold codes, or whatever), you wouldn't need pilots, right? Couldn't you do fine carrier adjustments on the sequences themselves? I guess you don't know when you're getting the sequence or the flipped sequence, but the idea of putting pilots inside the m-sequence seems kind of strange, and the sequence seems to have really good timing information anyway. I suppose you could put pilots between subsequent bits though.

This is sadly an area where my DSP knowledge falls short. However, I'm fairly close to what I think is a working implementation without fine carrier adjustments, and I'll probably ask for feedback soon :)

[jgaeddert]

Yes, you should be able to perform adjustments on the sequences themselves, even if they're modulated by an unknown e.g. QPSK symbol, you should be able to track carrier phase/frequency and timing.

Feel free to ask for feedback. That's why I'm here! =)

[brian-armstrong]

Thank you, I really appreciate how helpful you've been. Your project has taught me so much about DSP.

I got my gen and sync working, sort of. My modulation scheme is simple, I just take the complex sequence from the m-sequence and multiply by -1 for '0's. The sync actually decodes values now, but I still haven't done fine freq acquisition. I have been playing with the example testing harness and as you might expect, this all falls apart once you add some frequency offset. I was actually surprised that even just adding noise causes issues since the coarse frequency lock induces a small frequency offset in the presence of noise in the preamble.

I've been looking at the values generated and the correlations in the synchronizer just to double-check that there's no major bugs, but it does seem like fine frequency acquisition is needed.

From what I can tell from qpilotsync, it looks like that synchronizer takes the full header, decimates it to just the pilots, and then derotates them against the known pilot sequence. It looks like then it uses an FFT to find the dominant frequency in the derotated sequence.

With my modulator, the '0's will be out of phase by 180 degrees, which I suspect wouldn't work with this kind of synchronizer. I can imagine an iterative solver that first demodulates the first symbol and then uses that to derotate, and so on for the rest of the sequence, but is there a better approach? Is this sort of synchronizer appropriate when the sequence isn't necessarily known?

[brian-armstrong]

Hm, I just remembered that the homebrew GPS page [1] describes how it handles this issue, which is to despread on an early sequence, a punctual sequence, and a late sequence, and then adjust the NCO accordingly. I think there might be enough detail here that I could try implementing this, though I suspect it's a fairly beefy synchronizer.

[1] http://www.aholme.co.uk/GPS/Main.htm

[brian-armstrong]

I played with that method some today and I think I finally understand the depth of the problem.

As far as I can tell, there are actually two sorts of offsets we're concerned with. The first is the traditional phase/freq offset induced by a mismatched sinusoidal carrier, which imparts a slow moving sine wave across the entire group of captured samples. The second kind of offset is a sample rate mismatch which causes the m-sequence to drift and eventually shift to an off-by-1 sequence which kills the correlative properties.

It looks like the link I gave is concerned with the second type of carrier drift, but qpilotsync deals with the first. And a mismatched sample rate will likely cause both kind of drifts.

I've convinced myself that derotating the received sequence and using the qpilotsync method will work as long as we use cabsf() on the derotated sequence. This throws away phase information, but I believe this will only lose high-frequency changes which coarse acquisition should have eliminated anyway.

The second kind of drift also seems like an issue but may not be as important to solve at first. I feel like a good synchronizer probably needs some system to fix both. If the sample clocks have a combined error of 1ppm then the off-by-1 drift should happen after 1 million samples * the interpolation rate.

Does this seem correct? Am I completely off? I am going to try the cabsf() qpilotsync-esque synchronizer first since that's the sort of error induced in the test harness anyway.

[brian-armstrong]

Sorry for the late update, but I just had a good bit of success. In order to fix the carrier offset, what I ended up doing was creating a PLL initialized to 1e-4 loop bandwidth and 0 initial freq/phase. Then I update the phase error with cargf(received symbol * conjf(msequence symbol)) (with a bit of mapping -- cargf near -PI and PI are flipped around). Mixing down with this pll seems to be all that's needed to track the fine carrier offset.

I haven't tackled the early/late gate yet, but I'm pretty excited to start testing this in the wild. In the sim, this setup seems decode pretty reliably at 0.05 carrier offset and -3.0dB SNR (of course, the actual bit energy to noise is much greater). I'm not sure what SNR I should expect to work, but at any rate what I'm really curious about is how it will do in my very echoey room.

[brian-armstrong]

I've finally got this mode hooked up in Quiet, and it looks promising so far. I am getting very good results when attempting the previous test that wasn't working with BPSK or OOK. I've even tried it in the next room over, and it seems to work every once in a while, which is fairly surprising to me. It's only 10 bits/s, but still, not bad.

[jgaeddert]

That's awesome! I had some thoughts for other options for physical layers you could try, but I'd be interested in seeing your current solution.

[JonMuehlst]

Is any of the code mentioned regarding DSSS available publicly?

[nalnasery]

I tried to search for a documentation regarding DSSS framegen/framesync and was not able to find one. Is there anything I could refer to in order to understand the example?

[jgaeddert]

Unfortunately I haven't gotten around to writing up the documentation for it yet. The framing structure itself is still a work in progress. Is there anything in particular you have questions about?

[nalnasery]

I am trying to understand the purpose of each STATUS inside dsssframegen_generate_symbol since it seems to be the one responsible for generating the input symbols in dsssframegen_write_samples. In other words, how does dsssframegen_generate_symbol and dsssframegen_write_samples work?

[FullyArticulate]

There's at least two bugs in the dsss code.

1) In dssframegen_getframelen(), the logic _q->frame_assembled somehow got flipped preventing getframelen being called after a frame has been assembled.

2) dsssframesync_example does not work. (Notice the callback is never called).

I've fixed the first issue in my own code, but it still isn't working. It detects the frame, goes to dsssframesync_configure_payload() which fails in the qpacketmodem_decode_soft_payload() call.

[jgaeddert]

Hmm... I'll take a look at this

[brian-armstrong]

It looks like this commit removed a bunch of braces, including one that was essential. e.g. https://github.com/jgaeddert/liquid-dsp/commit/cd71c43219e9fa5aa8f1e7d5c4f7a343826199ac?branch=cd71c43219e9fa5aa8f1e7d5c4f7a343826199ac&diff=unified#diff-a97c6b42a0582f8c44d12e54d3552d774f9fac2868b7cccf4b266492cc723f9bR408

(You'll have to click 'expand diff' for dsssframesync.c after clicking this link, then visit the link again to get to the specific line)

In that same block, it also removed an essential closing brace on what used to be L 419. So the whole block there has different logic than it used to.

[brian-armstrong]

You can actually see this where the full block is unreachable https://github.com/jgaeddert/liquid-dsp/blob/master/src/framing/src/dsssframesync.c#L400

[brian-armstrong]

In general, braces are your friend. There are some fairly famous examples of bugs caused by not having them, like this Apple security issue from 2014 https://embeddedgurus.com/barr-code/2014/03/apples-gotofail-ssl-security-bug-was-easily-preventable/

[brian-armstrong]

Some other issues I notice with this commit as I look at it

I have found liquid_float_complex is decidedly nicer to work with than float complex because it can make it easier to work with C++ compilers on liquid. There are some good reasons to compile liquid as C++ rather than C, namely for MSVC. By using liquid_float_complex everywhere, you can just typedef it to std::complex and be done, full compat achieved. There might also be some other interesting possibilities for platforms that don't strictly provide float complex.

For convenience's sake, calling destroy() on a NULL feels like it should be a success. This makes the caller have to care a bit less about what it destroys. I suppose this depends on what the semantics are exactly but if you compare to, say, free()ing a NULL in C, that is perfectly legal. The only thing that isn't legal is trying to destroy something that looks like a valid pointer but doesn't actually point to an instance of the thing. Though the caller may not actually look at this result anyway.

[jgaeddert]

Well, this serves me right for not doing adequate testing...

[FullyArticulate]

Yeah, how dare you provide an insanely useful library for free and not make it work perfectly for everyone.

[JayKickliter]

Joking aside, this is hands-down the best documented and most consistent library I've ever encountered.

[FullyArticulate]

There still lurk some other bugs in the DSSS code. I can now reproduce at will where framesync perfectly finds the start of frame and all the correct timing parameters, but the header and payload will never decode again after a few hundred packets at 500khz. I'm diagnosing, but no luck yet... It seems that low snr causes it to "run off the rails". I leave the receiver running and restart the transmitter at a very high power, and the receiver still finds the sync perfectly, but cannot decode the header or payload.

[campagn1]

I am also interested in this: I am using dsssframesync_example with no success. Here a first bug:

int dsssframesync_execute(dsssframesync _q, float complex * _x, unsigned int _n)
{
    unsigned int i;
    for (i = 0; i < _n; i++) {
        switch (_q->state) {
        case DSSSFRAMESYNC_STATE_DETECTFRAME:
            // detect frame (look for p/n sequence)
            return dsssframesync_execute_seekpn(_q, _x[i]); ...

in this function the for is useless due to the return in each switch case. To keep the signature consistent to flexframe perhaps two function int dsssframesync_execute(dsssframesync _q, float complex _x); and int dsssframesync_execute_block(dsssframesync _q, float complex * _x, unsigned int _n) can be used. Still, even fixing this fix, the result is that the frame is correctly detected, but header's crc fails, and so does payload's crc.

In particular, _q->header_valid is always 0 when dsssframesync_configure_payload is called

Other point: beta is not the same for dsssframesync.c and dsssframegen.c: is this correct?

BTW, thank you making this framework available, it's amazing!

[seonlee2013]

好啊，邮件收到啦！ Hi, your email is arrived safely

[brian-armstrong]

I am also interested in this: I am using dsssframesync_example with no success. Here a first bug:

int dsssframesync_execute(dsssframesync _q, float complex * _x, unsigned int _n)
{
    unsigned int i;
    for (i = 0; i < _n; i++) {
        switch (_q->state) {
        case DSSSFRAMESYNC_STATE_DETECTFRAME:
            // detect frame (look for p/n sequence)
            return dsssframesync_execute_seekpn(_q, _x[i]); ...

in this function the for is useless due to the return in each switch case. To keep the signature consistent to flexframe perhaps two function int dsssframesync_execute(dsssframesync _q, float complex _x); and int dsssframesync_execute_block(dsssframesync _q, float complex * _x, unsigned int _n) can be used. Still, even fixing this fix, the result is that the frame is correctly detected, but header's crc fails, and so does payload's crc.

In particular, _q->header_valid is always 0 when dsssframesync_configure_payload is called

Other point: beta is not the same for dsssframesync.c and dsssframegen.c: is this correct?

BTW, thank you making this framework available, it's amazing!

It looks like this broke in some refactor PR. https://github.com/jgaeddert/liquid-dsp/commit/cd71c43219e9fa5aa8f1e7d5c4f7a343826199ac#diff-a97c6b42a0582f8c44d12e54d3552d774f9fac2868b7cccf4b266492cc723f9bL286

I believe it likely worked previously. It sounds like the PR might not have been tested. Maybe try an earlier version of the library to see if that fixes it.

[campagn1]

Thank you a lot!! In v1.3.2 I got the output:

******** callback invoked
    EVM                 :     0.00000000 dB
    rssi                :     0.00000000 dB
    carrier offset      :     0.00000000 Fs
    num symbols         :   0
    mod scheme          :   unknown (0 bits/symbol)
    validity check      :   crc32
    fec (inner)         :   none
    fec (outer)         :   none
    header crc          :   pass
    payload length      :   20
    payload crc         :   pass
    payload bit errors  :   0 / 160
done.

in this version dsssframesync_execute was a void function and did not return inside the switch case, making the for cycle working as expected. Changing this in the last version makes the dsss example working as well.

Also in v1.3.2 q->beta = 0.25f; in dsssframegen.c and q->beta = 0.3f; in dsssframesync.c: is this correct?

In the case you make the fix, I would also suggest to remove printf("seeking pn...\n"); and printf("FRAME DETECTED\n"); that slows down the code.

I guess the afroementioned error in dsssframegen_getframelen() is in the first if statement: check flexframegen_getframelen() implementation. Cheers, Filippo