daniestevez / gr-satellites

GNU Radio decoder for Amateur satellites
GNU General Public License v3.0
769 stars 160 forks source link

AALTO-1 CC1125 frames #210

Closed janvgils closed 3 years ago

janvgils commented 3 years ago

Triggered by the development that is being done by SA2KNG where he tries to integrate gr-satellites decoding into SatNOGS we discovered that the gr-satellites AALTO-1 decoding seems to no longer produce data.

For this test I have used the following two observations:

https://network.satnogs.org/observations/3315578/ https://network.satnogs.org/observations/3315127/

The SatNOGS decoder in combination with observation 3315127 gives the following output:

10 20 2E 07 0D 00 CD B9 34 6E AD A8 5A A2 49 65 21 A3 36 B7 D6 16 67 D6 2A 45 B5 AE C6 82 58 78 60 23 2A 56 C0 5E 68 9F 0F 24 9E 51 5B 70 96 90 4B C6 5A 21 65 C2 73 EA 6C 4D EC 56 3E 7E EB 98 97 79 02 AD 23 D6 B5 2F 59 F6 13 FE EE 2B 81 54 6B C1 0D 5B A5 6D 20 B5 C2 F8 26 21 97 AC 15 27 E5 84 27 D4 EE FA 4E BF E3 5A 06 C7 6A 0B 22 FD F2 2B 4B 5F C2 A7 F9 12 B3 2C 1D A1 AD 41 C5 FE B8 58 C8 F5 2C 03 AC 15 2B 05 F8 C6 7A 96 B1 D9 25 0C C7 EA 81 62 5D AF 11 C5 90 66 DF A1 62 05 BC 0B 6D 28 15 2B 79 B1 90 4B C3 F1 72 29 C0 DF 55 BD 39 56 26 36 5C B7 C7 B0 40 51 AC EF 9A C7 36 AC 4D D3 89 5A DE B0 C0 F6 16 C9 BA 7D 73 FA 65 3A 32 6B 0D E5 9C F6 44 91 EF 7C 22 0E 2E 49 B4 8A 7C 2E 38 D7 F5 B8 CB D8 CC D8 20 6C E1 1B 04 55 DC C2

Using the following gr_satellites command with the audio from this observation gives the following output:

gr_satellites AALTO-1 --wavfile Recordings/OGG/AALTO-1_satnogs_3315127_2020-12-18T07-10-46.wav --samp_rate 48e3

-> Packet from 9k6 FSK downlink
Could not parse telemetry beacon Error in path (parsing) -> header -> addresses -> callsign
stream read less than specified amount, expected 6, found 0

When using the other observation we have a similar output:

gr_satellites AALTO-1 --wavfile Recordings/OGG/AALTO-1_satnogs_3315578_2020-12-18T08-42-53.wav --samp_rate 48e3

> Packet from 9k6 FSK downlink
Could not parse telemetry beacon Error in path (parsing) -> header -> addresses -> callsign
stream read less than specified amount, expected 6, found 1

When I use the example from the satellite-recordings repository the decoding works fine.

There is one addition that can maybe help in solving this, the SatNOGS transmitter definition seem to rely on some "Rktr" framing.

The AALTO-1.yml file uses AX.25 G3RUH framing, I guess it is possible something has changed, maybe the following information on there website already points to a possible explanation:

Telemetry/Telecommand exchange over Aalto Ground Station (OH2AGS) using TI CC1125 packet protocol

  • Preamble and sync word: 0x35 0x2E 0x35 0x2E
  • Data and symbol rate: 9600 bits/s
  • Variable length packets
  • Protocol specification: TBD

AALTO-1 satellite information: https://wiki.aalto.fi/display/SuomiSAT/Radio+Amateurs

daniestevez commented 3 years ago

Hi Jan, thanks for reporting this. The problem we find here is that the telemetry parser is getting frames that don't match the format it expects. In this cases, it helps to run with --hexdump to see what are the frames that are being decoded. It also helps to look at the SatYAML file to see what is the telemetry parser for the satellite. In the case of AALTO-1 it is the generic ax25 parser, which only parses the AX.25 header. So whatever that is arriving to the parser doesn't look like an AX.25 header.

I'll treat the two observations separately, because I think they are very different.

Observation 3315578

Here I think that the problem is that the transmitter mode is "TLM CW", so the OGG file doesn't have the format we expect for an FSK signal. I wonder if the FSK signal could be recovered from this OGG file, but probably not, because the CW mode sets a too narrow bandwidth.

If we run with --hexdump we get

 $ gr_satellites AALTO-1 --samp_rate 48e3 --wavfile satnogs_3315578_2020-12-18T08-42-53.wav --hexdump
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK downlink))
pdu_length = 1
contents = 
0000: bd 
***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK downlink))
pdu_length = 12
contents = 
0000: 50 a9 0d f4 42 ef 82 ab bd d6 63 8c 
***********************************

These are garbage decodes (not so unlikely with AX.25), and their length is shorter than an AX.25 header, which explains the problem.

Observation 3315127

Here the mode for the SatNOGS observation is set correctly to GFSK9600. However if we run gr-satellites with --hexdump we see the following garbage decode only:

 $ gr_satellites AALTO-1 --samp_rate 48e3 --wavfile satnogs_3315127_2020-12-18T07-10-46.wav --hexdump
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK downlink))
pdu_length = 7
contents = 
0000: 40 cc f0 3e ac 18 a0 
***********************************

Again, this doesn't have the required length for an AX.25 header.

What is the problem with this file. If we look at it in Audacity, we'll see that it is badly saturated and clipped. Even the packets themselves (which have lower amplitude than the noise when there is no packet) have some clipping. This deteriorates the signal quality. Maybe it would be possible to tweak the decoder parameters by hand (--clk_bw, --clk_limit) to try to get a decode (this I haven't tried). This problem with clipping is something that I reported several months ago. Not sure if they've deployed the fix already.

Finally, the decodes that appear in SatNOGS look like garbage to me. The AX.25 headers doesn't seem correct (though I haven't run them through a parser to check this). I've also noticed that the mode is listed as " GFSK Rktr 9600", which seems to refer to Reaktor Hello World, which uses something quite different from AX.25.

janvgils commented 3 years ago

Thanks for the comment Daniel,

In the meantime, I have also received some IQ recordings and I will have a look at them, the recordings are made with the help of the SatNOGS IQ save option and this means we need to convert them, I am in the middle of it.

The first result aren't producing any data, the files can be download at following location.

gr_satellites AALTO-1 --rawfile Recordings/IQ/AALTO-1/3252405_2020-12-06T07-57-38_AALTO-1.raw --iq --samp_rate 48e3 --hexdump

gr::log :WARN: file_source0 - file size is not a multiple of item size

Based on the error output I also need to look in to the conversion.

kng commented 3 years ago

Those recordings would be with sample rate of 57600. Also a lot faster to bypass the throttle if you only need to convert the files. The original .raw can be used directly with gr_satellites using --iq --rawint16

daniestevez commented 3 years ago

To add to what @kng said (which is basically how I would try to run this), I will add that the

gr::log :WARN: file_source0 - file size is not a multiple of item size

warning is harmless. This is probably caused by the last few bytes of the file being chopped off as the recording stopped.

janvgils commented 3 years ago

I tried multiple --samp_rate values and this changes the output data, below the output based on --samp_rate 57e3.

for a in Recordings/IQ/AALTO-1/*.raw
 do 
  echo $a
  gr_satellites AALTO-1 --iq --rawint16 $a --samp_rate 57e3 --hexdump
done

And that is producing the following output

Recordings/IQ/AALTO-1/3293703_2020-12-14T08-26-05_0_AALTO_1.raw

* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK downlink))
pdu_length = 7
contents = 
0000: 88 bc c5 3c 20 c0 2c 
***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK downlink))
pdu_length = 33
contents = 
0000: c0 78 bc b9 21 0a 6d a6 5f 20 6a 9d ff 60 2c 31 
0010: e8 90 e4 e0 10 22 6b 39 9d 89 18 eb 43 1b 99 3e 
0020: 02 
***********************************
Recordings/IQ/AALTO-1/3312221_2020-12-17T16-42-42_AALTO-1.raw
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK downlink))
pdu_length = 1
contents = 
0000: 80 
***********************************
kng commented 3 years ago

I've replayed 3312221 in gqrx with mode nbfm (sending it to gr_satellites in --udp mode, without results) and it looks like two different modulations and also shifting frequency. Perhaps someone with good eyes/ears can identify the modulation type. https://imgur.com/a/qffEfSX

janvgils commented 3 years ago

Here some info I got from a QT GUI Time sink.

The following OBS was used: https://network.satnogs.org/observations/3316321/

Where do we start with counting the bits?

daniestevez commented 3 years ago

I've replayed 3312221 in gqrx with mode nbfm (sending it to gr_satellites in --udp mode, without results) and it looks like two different modulations and also shifting frequency. Perhaps someone with good eyes/ears can identify the modulation type. https://imgur.com/a/qffEfSX

That seems to me like two different satellites. The modulations are different (they're both FSK, but the deviation/baudrate seems different and one of the has long preambles and postambles), the signal strength is different and the centre frequency is not quite the same.

daniestevez commented 3 years ago

@janvgils thanks for that plot. We should count the bits after the 0101010101 preamble. Some kind of syncword would usually follow afterwards. However it's sometimes a bit difficult to tell exactly where the preamble starts and the syncword starts.

At first I wasn't able to recognize this syncword as any of the usual suspects (it's definitely not AX.25). Then I thought about the fact that some SatNOGS observations were using "Reaktor" as the mode, reviewed my [notes for Reaktor Hello World], and realized that the syncword in your plot seems to match the syncword 0x352E352E used by Reaktor Hello World.

I'm wondering what is happening here. I think that probably it's possible to use the Reaktor Hello World deframer from gr-satellites to decode these frames.

daniestevez commented 3 years ago

Now I read again the information you pasted about the "Telemetry/Telecommand exchange over Aalto Ground Station (OH2AGS) using TI CC1125 packet protocol" and realize this is what we are seeing. Reaktor Hello World uses the same chip, and also the same syncword (which is probably a default value).

However on my first try the Reaktor Hello World deframer doesn't work correctly with these frames. The syncword is detected correctly, but the CRC check fails.

Probably there is some variation in the packet framing.

daniestevez commented 3 years ago

So the packets use the typical PN9 whitening of the CC11xx, seem to contain a length field, have a payload that consists mostly of ASCII text like this:

/a1/radmon[32389]: RADMON scientific data, ts 1608144390, read 56, status=cc
/a1/radmon[32389]: RADMON housekeeping, ts 1608144390, read 53, status=0
/a1/radmon[32389]: RADMON ec, ts 1608144390, read 22, status=/
/a1/radmon[32389]: RADMON scientific data, ts 1608144405, read 56, status=cc
/a1/radmon[32389]: RADMON scientific data, ts 1608144420, read 56, status=cc

and they also seem to have a CRC-16. However I can't check the CRC with the CC11xx CRC checker, for some reason.

Here is the flowgraph I'm using to play with this: aalto.grc.zip

janvgils commented 3 years ago

The wonderful world of satellite decoding, the example wav file that is part of the satellite-recording repository is way different.

I used the flowgraph you shared and produces the following output, I have no idea if there is still a ground station active that could share telemetry information.

***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
()
pdu_length = 245
contents = 
0000: f2 10 29 7e 07 0d 41 31 6c 6f 6e 5b 33 32 33 38 
0010: 39 5d 3a 20 52 41 54 49 46 48 2c 73 7b 69 65 6e 
0020: 74 69 a6 69 63 20 6c 25 74 41 2c 20 f4 73 60 31 
0030: 36 36 18 f1 34 16 30 31 36 6d 00 70 75 60 64 20 
0040: 35 26 2d 20 70 d4 61 f4 75 53 39 63 63 2a 2f 71 
0050: 35 2f 73 e1 66 6d 7e 6e 5f 33 32 13 38 39 5d 3a 
0060: 20 52 41 40 4d 4f 6e 20 68 6f 75 7b ed e3 65 e5 
0070: 70 68 ae 67 2c 26 74 73 20 d1 36 30 58 f9 34 36 
0080: 30 31 3b 2c 20 72 65 61 65 21 34 30 2c 2c 73 74 
0090: 67 6c 65 73 2d 30 0a 2f 69 31 2f f2 21 64 6d 6f 
00a0: 6e 5b 13 32 33 38 39 5d 39 00 52 41 44 5d 4b 4f 
00b0: 20 65 02 2c c0 74 73 20 01 37 b0 7e 35 30 36 b8 
00c0: 31 32 2c 10 77 63 61 44 20 32 32 20 20 73 74 61 
00d0: 54 75 73 3d 31 31 0e 2f 60 21 2f 72 61 64 6d 75 
00e0: 6e 73 32 32 73 38 38 dd 3a 20 52 41 41 0d 4f 4e 
00f0: 20 77 63 af 9d 
***********************************

Then we are only left, with the mystery, where a satellite is changing from AX.25 G3RUH framing to Reactor World

daniestevez commented 3 years ago

Hi Jan,

I have still some open questions regarding this satellite.

The change in framing is clear to me. As shown in their webpage, they broadcast an AX.25 worldwide, because that's perhaps considered easier for hams to receive, and use the native CC1125 protocol when over their groundstation.

kng commented 3 years ago

I also get short frames obs 3252405, with the second and third byte looking like a counter.

From: 3316321 Pretty cool (:

Power up
PDM 1 enabled
PDM 6 enabled
Send configuration
/a1/radmon[32383]: starting the Aalto-1 RADMON mission
timeout! dataleft: 87, total length: 88
Failed to received response!
/a1/radmon[32383]: RADMON send cfg, ts 1608138668, read -1, status=0
/a1/radmon[32383]: An error occured with RADMON send cfg, status: 0x0
/a1/radmon[32383]: RADMON send cfg, ts 1608138668, read 0, status=0
Sending configuration..
Success
Start measuring
/a1/radmon[32389]: starting the Aalto-1 RADMON mission
/a1/radmon[32389]: RADMON send cfg, ts 1608138669, read 0, status=0
/a1/radmon[32389]: RADMON scientific data, ts 1608138669, read 56, status=cc
/a1/radmon[32389]: RADMON scientific data[J, ts 1608138684, read 56, status=cc
/a1/radmon[32389]: RADMON housekeeping, ts 1608138684, read 53, status=0
kng commented 3 years ago
* Are the decoded frames in [SatNOGS observation 331527](https://network.satnogs.org/observations/3315127/) correct? These used "GFSK Rktr 9600". They don't seem to have ASCII data, but they might be correct nevertheless. If so, what is SatNOGS doing to decode these?

One of the last frames on that obs have this /a1/radmon_16-12-20_171001.dat.gz

janvgils commented 3 years ago

Good afternoon Daniel,

I have still some open questions regarding this satellite.

Lets see if I can find some information,

  • What is the CRC? I wasn't able to use the same CRC check as for Reaktor Hello World, so I'm guessing there is some small difference. Maybe it would be possible to contact the AALTO-1 team and ask them.

I will try and find some additional information by contacting the team, if they are still available.

  • Are the decoded frames in SatNOGS observation 331527 correct? These used "GFSK Rktr 9600". They don't seem to have ASCII data, but they might be correct nevertheless. If so, what is SatNOGS doing to decode these?

@kng Can you help with the question regarding the SatNOGS decoder?

petrinm commented 3 years ago

Hi, I'm one of the Aalto-1 team members who is still somehow in the scene and knowledgeable about how the satellite was build so I could bring some clarity to this discussion. First of all, Aalto-1 transmits two different frame formats. 1) The beacon CW/morse (with ugly side bands) with a G3RUH AX.25 frame immediately after the CW transmission. 2) Telemetry and telecommanding frames which follows the CC11xx formatting. The formatting specs in a nutshell are: modulation 2GFSK, datarate 9600 bits/s, deviation 3.17kHz, syncword: 0x35 0x2E 0x35 0x2E, data is scrambled using TI's PN9 scrambler/whitening, the first byte in the frame is the length byte and in the end of the frame there's a CRC16 checksum. The CRC16 is same as for the AX.25 frames and it's not the one implemented by the CC11xx. The frame formating is very similar to Reaktor's Hello World formatting and their radio hardware is actually based on Aalto-1's radio design. The largest differences are that the Hello World uses CC11xx's own CRC16 implementation and inside the radio frames they have CSP-frames. More about Aalto-1's telemetry data can be read also from this post: https://community.libre.space/t/decoding-packets-from-aalto-1-42775/5944/6

The Aalto-1 satellite is starting to be at the end of its lifetime. Atm, it's still being used as a tool for space technology teaching and its main instruments - AaSI and RADMON - are being operated somewhat frequently. Right now during this Christmas, Aalto-1 will be measuring using the RADMON instrument and its file data dumps can be heard during the passes over Finland. Similar file data dumps can be also seen in the SatNOGs observation mentioned above. Most of the data is in binary format and also compressed using gzip. Thus, filenames like /a1/radmon_16-12-20_171001.dat.gz can be spotted in the downlink frames. Due to the nature of the transmitted blocks of compressed binary data, single frames don't convey that much of understandable information but sometimes we also downlink some of the text logs as uncompressed ASCII for debugging purposes. That's what @kng was able to decode. :) More about the RADMON instrument and its results can be read for example from this paper: https://www.researchgate.net/publication/337336871_Radiation_Monitor_RADMON_aboard_Aalto-1_CubeSat_First_results

janvgils commented 3 years ago

Petri,

Thank you very much for sharing this information.

More about Aalto-1's telemetry data can be read also from this post: https://community.libre.space/t/decoding-packets-from-aalto-1-42775/5944/6

Is the full telemetry scheme publicly available?

daniestevez commented 3 years ago

Thanks @petrinm for all the information. The knowledge that the CRC is the CRC-16 CCITT used in AX.25 has saved me some trial and error to find the correct CRC algorithm.

To clear things up: I don't think that any of the decoders have suddenly stopped working or that the frames transmitted by AALTO-1 have changed. To this date, gr-satellites has only supported decoding the AX.25 frames from AALTO-1. I think that what has happened is that @janvgils and @kng have "discovered" the CC1125 frames that AALTO-1 transmits only over Finland and wondered what they were. I'm not sure about SatNOGS, but it seems that it has the decoder for CC1125 already working since some time ago.

I have now implemented a decoder in gr-satellites for the CC1125 frames.

Regarding the data dumps transmitted during this Christmas, if these are file chunks with a header indicating the chunk number, etc, I think that it would be quite easy to use the gr-satellites FileReceiver framework to add support for reassembling these files in gr-satellites. For this, I would need to know the format of the chunk header.

janvgils commented 3 years ago

Good evening Daniel,

I guess with the latest gr-satellites update this issue can be closed.

gr_satellites AALTO-1 --wavfile Short_AALTO-1_satnogs_3316321_2020-12-18T19-28-51.wav --samp_rate 48e3 --hexdump

* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK CC1125 downlink))
pdu_length = 38
contents = 
0000: 10 28 2b 07 08 1a d4 13 2f 61 31 2f 72 61 64 6d 
0010: 6f 6e 5f 31 36 2d 31 32 2d 32 30 5f 31 37 31 30 
0020: 30 31 2e 6c 6f 67 
***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK CC1125 downlink))
pdu_length = 38
contents = 
0000: 10 28 2c 07 08 1a d4 13 2f 61 31 2f 72 61 64 6d 
0010: 6f 6e 5f 31 36 2d 31 32 2d 32 30 5f 31 37 31 30 
0020: 30 31 2e 6c 6f 67 
***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK CC1125 downlink))
pdu_length = 38
contents = 
0000: 10 28 2d 07 08 1a d4 13 2f 61 31 2f 72 61 64 6d 
0010: 6f 6e 5f 31 36 2d 31 32 2d 32 30 5f 31 37 31 30 
0020: 30 31 2e 6c 6f 67 
***********************************
* MESSAGE DEBUG PRINT PDU VERBOSE *
((transmitter . 9k6 FSK CC1125 downlink))
pdu_length = 242
contents = 
0000: 10 28 2e 07 0d 00 00 46 69 6c 65 20 73 77 61 70 
0010: 20 31 36 2d 31 32 2d 32 30 5f 31 37 31 30 30 32 
0020: 0a 50 6f 77 65 72 20 75 70 0a 50 44 4d 20 31 20 
0030: 65 6e 61 62 6c 65 64 0a 50 44 4d 20 36 20 65 6e 
0040: 61 62 6c 65 64 0a 53 65 6e 64 20 63 6f 6e 66 69 
0050: 67 75 72 61 74 69 6f 6e 0a 2f 61 31 2f 72 61 64 
0060: 6d 6f 6e 5b 33 32 33 38 33 5d 3a 20 73 74 61 72 
0070: 74 69 6e 67 20 74 68 65 20 41 61 6c 74 6f 2d 31 
0080: 20 52 41 44 4d 4f 4e 20 6d 69 73 73 69 6f 6e 0a 
0090: 0a 74 69 6d 65 6f 75 74 21 20 64 61 74 61 6c 65 
00a0: 66 74 3a 20 38 37 2c 20 74 6f 74 61 6c 20 6c 65 
00b0: 6e 67 74 68 3a 20 38 38 0a 46 61 69 6c 65 64 20 
00c0: 74 6f 20 72 65 63 65 69 76 65 64 20 72 65 73 70 
00d0: 6f 6e 73 65 21 0a 2f 61 31 2f 72 61 64 6d 6f 6e 
00e0: 5b 33 32 33 38 33 5d 3a 20 52 41 44 4d 4f 4e 20 
00f0: 73 65 
***********************************

Big thanks to all that made possible.

daniestevez commented 3 years ago

Thanks Jan for testing this, and again huge thanks to @petrinm for making this possible. I'll now be closing this, since the decoder seems to work correctly and all loose ends are explained.