BatchDrake / ccsds-tc

BCJR-based decoder for CCSDS turbo codes (according to CCSDS 131.0-B-3)
17 stars 1 forks source link

Input File questions #1

Open K4KDR opened 2 years ago

K4KDR commented 2 years ago

Thank you SO much for making this available!!

Regarding the "32-bit complex float samples" as input... would an example of the required input be the 32-bit .RAW format that GQRX saves as I/Q recordings?

If not, by what means would you recommend that a input file be created in the proper format?

Also, is there a sample file available somewhere that we can use for testing (to verify that we have the software installed properly)?

Thanks again!

-Scott, K4KDR

BatchDrake commented 2 years ago

Regarding the "32-bit complex float samples" as input... would an example of the required input be the 32-bit .RAW format that GQRX saves as I/Q recordings?

Yes and no. That's indeed the data format, but Gqrx is not able to perform this demodulation. Instead, you would need to use either SigDigger or GNU Radio to perform the demodulation of the BPSK subcarrier. If you are using EA4GPZ's GNU Radio flowgraph, ccsds-tool fits right between the Costas loop and the complex-to-real block:

lucy_ata

If you use SigDigger, there are two scenarios. If your SNR is high you could try to lock onto the BPSK subcarrier and forward samples either to a file or directly to a socket. This is how you'd do it:

  1. Put the filterbox in one of the subcarriers (both transmit the same information) and adjust its bandwidth:

Captura de pantalla de 2021-11-09 10-19-13

  1. In the Inspection tab, tick PSK Inspector and click in Open Inspector. Go to parameter estimation, non-linear baud estimator, click on Estimate. This will start to guess the baudrate of the BPSK signal (and as a first estimation, it is usually rather accurate). As soon as the value stabilizes, click on Apply and the go to Demodulator controls. You will see that this value (62.455 kbps in my case) has been copied to the baudrate field inside the Clock recovery box.

cyclo

  1. There is a wide range of demodulator parameters that may work here, but for instance, in my case, I enabled AGC, set the AFC to Costas/BPSK, matched filter to SRRC with maximum roll-off, and set Gardner clock recovery with around -24 dB loop gain (don't mind the spectrum). I can tell the BPSK demod has recovered the carrier because the histogram above is not flat:

demod

  1. Start ccsds-tool in listening mode in a separate terminal. Just run ccsds-tool -l 40404 to accept connections in the TCP port 40404. Since the rate is rather low, you may also want to launch it as stdbuf -o0 ccsds-tool -l 40404 to output the frames right after they are decoded. In this example, I am also redirecting the output to hexdump -C just to view the decoded frames, but you may want to store them to a file instead:

Captura de pantalla de 2021-11-09 10-55-35

  1. In SigDigger, go to the Data forwarding tab, in Data properties set the Variable to Baseband I/Q data. Leave network forwarding controls as-is (or adjust the host/port if you chose to run ccsds-tool in a different machine) and click on Forward. If everything works, you should see decoded frame data in hexadecimal form:

capture

Note the rate is rather low (60 kbps at 1/6 means 10 kbps of useful data, which in turn translates to roughly 1 8920-bit packet per second). If you fail to see data at this rate, maybe you need to tweak the demodulator controls a bit. What I usually do is to use the SigDigger's Symbol stream tab to look for patterns in the demodulated symbols and see whether the clock recovery is not missing samples.

If your SNR is low (which is the case for the signal I dealt with), well, you may have trouble making SigDigger lock onto the subcarrier. Nonetheless, for AM signals and good PLL lock on the central carrier, there's a trick in which I can get a 3 dB SNR boost by overlapping both subcarriers (this is mathematically simpler than it looks). The problem is that this does not work out of the box (at least for now): you have demodulate the signal as AM, reduce the PLL bandwith to a point in which it synchronizes even in the presence of strong noise power (in my case, I had to reduce it down to 20 Hz), save the I/Q samples of the synchronized signal to a file, run a script to add (or subtract) every complex sample to its complex conjugate (this is what produces the 3 dB boost), and then open SigDigger again on the corrected file.

Of course, I am not going to ask you to do all this. I am working on making SigDigger be able to deal with "signals inside signals" (i.e. AM signals with PSK subcarriers like this one, FM signals with AM subcarriers like APT, FM signals with PSK subcarriers like RDS...) so you can do all this from the same UI.

PS: if you ask what's the point of all this, I know this does not look too useful but we have valid (CRC-checked) digital frames here, so it's just matter now of deciding how much upper-layer decoding logic we want to implement (and with what purpose).

PS2: You can download from my website the samples-LLR.raw file (which has some BPSK-demodulated samples that you can pass directly to ccsds-tool and extract frames) and the 3db.raw I used to make the SigDigger screenshots of this post.

K4KDR commented 2 years ago

Thank you SO much for the detailed reply!!

I will get to work testing both with GNU Radio and also with your sample files. Both sets of instructions are very helpful and MUCH appreciated!

-Scott

K4KDR commented 2 years ago

Thanks again for the great help!

Regarding the 'output' file option, what format does that produce, exactly? Or perhaps asked more directly, once I run a command such as:

./ccsds-tool -f ../sample-files/samples-LLR.raw -o ../sample-files/samples-LLR.bin

... other than viewing the resulting .bin file with a hex editor, is there a "correct" way to view the decoded frames in the way that we, for instance, might view AX.25 packet decodes in Direwolf?

Thanks!

BatchDrake commented 2 years ago

The output are the raw bytes of the CCSDS frames, one after the other, with no additional metadata. Since in this case the frame length was 8920 bits, the data in the file consist of consecutive chunks of 8920/8 = 1115 bytes. I have not written anything specific to parse them (yet), but it should not be too difficult to inspect their headers. For instance, in my case, the first frame in the output file starts by:

4c 7f 00 d0 fb 00 07 fe aa aa ...

Or, in binary:

0100 1100 0111 1111 0000 0000 1101 0000 1111 1011 0000 0000 0000 0111 1111 1110 1010 1010 1010 ...

If you go to the standard (https://public.ccsds.org/Pubs/732x0b2c1s.pdf, page 4-2, figure 4-2: transfer frame primary header) you can group these bits into fields:

01 00110001 111111 |00000000 11010000 11111011| 0 0 00 0000 [0000 0111 1111 1110] 1010 1010 1010 ...
 V CRAFT_ID VCHAN            VCHAN FRAME COUNT  R C  S CYCL    HEADER ERROR CTRL  PAYLOAD

Since the virtual channel ID is 1111112=63, this is actually an idle frame (note the alternating bit pattern 010101...). I hope to release something to show the contents of the frames soon.

BatchDrake commented 2 years ago

Answering your question, there may be a way using Wireshark (https://www.wireshark.org/docs/dfref/c/ccsds.html) but I will probably have to write something to convert this dump into pcap/pcapng format (which is what wireshark actually understands)

K4KDR commented 2 years ago

Oh that is a HUGE help - thank you! Many of us have only had dealings with packets that were either in AX.25 format with 'Source' & 'Destination' Amateur Radio-style call signs or packets that were in no particular format that required that the telemetry format be known from the people who programmed the satellite. So, for me at least, this 'ccsds' format is new territory.

Thank you!

BatchDrake commented 2 years ago

Hi again @K4KDR

I just wanted to let you know that I posted yesterday an article on how to use SigDigger to do the demodulation directly, in real time, without intermediate steps like saving the demodulated data to a file:

https://www.reddit.com/r/SigDigger/comments/qwz1c0/amateur_dsn_with_sigdigger/

This refactor was long due and this issue you opened was the last push I needed to make it real. Hope it helps!

K4KDR commented 2 years ago

Thanks a million! Will read through that word-by-word.

Appreciate you taking the time to do that!