rs1729 / RS

radiosonde decoding
GNU General Public License v3.0
170 stars 56 forks source link

M10 serial number #7

Closed Viproz closed 4 years ago

Viproz commented 5 years ago

I am working on a port of your M10 code for radiosonde_auto_rx and to upload to APRS we need a 9 char identifier and you seem to have some knowledge about M10s serial number (https://github.com/rs1729/RS/blob/master/m10/m10ptu.c#L687) so I wanted to have your input.

From what I understand from your code the first number you generate is the sonde type, a hardware information (803 for ptu and 705 for gtop) the second number is always 2 ? and the last seems to be the serial number of the sonde. Do you have more knowledge of what theses numbers are ? Are you generating the same serial number as official people have ?

Thank you for your work !

rs1729 commented 5 years ago

I had only a limited number of radiosondes to compare with, so there was some guesswork/uncertainty about the middle part. Some radiosonde hunters from france had more samples and could confirm so far. In https://github.com/rs1729/RS/blob/master/m10/m10_msp430.txt#L114 you can also find the SN. That is 02 08 3a 31 25 in raw data and "310 2 11329" on the radiosonde.

I believe I only know M10s with the number 2 in the middle. (I don't remember if someone told me about an old M10 or M2K2(?) with a different number.) The first number should be the year and month (e.g. 310 -> 2013/10). For the last number, the M10/trimble all started with 1xxxx. I found one M10/gtop "704 2 23023", and there was 02 1a 74 cf 4b in the data. Thus gtop could be 2xxxx. So the xxxx digits (13 bits of the last two bytes (LSB) in raw data) are probably the serial for that year/month.

rs1729 commented 5 years ago

If you want to identify the sonde type, the 1st byte in the modem-frames is the frame length and 2nd byte should be the sonde type like this: ` frame byte[0..1]: byte[0]=framelen-1, byte[1]=type(8F=M2K2,9F=M10,AF=M10+)

M2K2 : 64 8F : 0110010010001111 M10 : 64 9F : 0110010010011111 (framelen 0x64+1) M10-aux: 76 9F : 0111011010011111 (framelen 0x76+1) M10+ : 64 AF : 0110010010101111 (w/ gtop-GPS) `

Viproz commented 5 years ago

Thanks a lot for all the infos, I hadn't implemented the auxiliary support yet because I didn't have any test audio but today at around 11 am UTC METOMODEM sent a M10 ptu with some auxiliary attachment from Ury so I was able to record a lot of data.

In my code I was using 0x649F20 to detect for a M10 ptu, the 64 is the length so I should use it another way but what about the 20 ? On ptu when they do their bip bip frame (the two close to each other) the ID is 0x64490c and the data is not in the normal format, do you know what they are transmitting during that frame ? Is the 20 and ID for the way we should decode the RS ?

Edit : I should add that during the bip bip event it is not always 0c at the end, I sometimes have 0b and I think 0a too

rs1729 commented 5 years ago

I choose the name 'ptu' for decoders with temperature output. Relative humidity is a little bit inaccurate because the sensor is temperature-dependent, so I didn't include that yet. And I don't know if there are M10 with pressure sensor. The classic M10 I would call M10-trimble, the new M10+ M10-gtop, the GPS is the main difference I believe. The decoding of the raw frames of M10 and M10+ is the same, so the 2nd byte could be used to do the different interpretations, if you include both in one program.

I don't know about the 3rd byte and the bip bip frame every 10 seconds, I think the new M10+/gtop doesn't transmit it anymore?! Maybe one could look at the firmware.

Since the frame starts with variable bytes (length, type), it cannot always be used to detect the header, maybe in a 2-pass processing. And everything before is more like a preamble that is similar to other radiosondes, so a multi-detect-tool could produce false-positives. You could do more post-processing, but this would slow down the detection-tool.

Viproz commented 5 years ago

Yeah my usage of PTU is probably wrong, I use it to identify what you call a normal M10. On their website METEOMODEM always say that the M10 do not have a pressure sensor and that they use altitude to calculate the pressure (probably in their software, not in the RS).

Yeah the checking is not a problem, I'm currently writing a better header detector that do not rely on a change in sign of the FM decoded signal, so far it's working good, I can detect more start of frame than with the other method but it still needs polish and a way to synchronize the start of the payload (I decide if a bit is 1 or 0 by taking the samples of two rawbits and comparing them against each others), there is no issue to treat the signal afterward with the good decoder. I just need to implement the auxiliary data (do you know if a gtop can have aux ?).

The M10+ doesn't transmit it indeed, I have a few recordings if you want Paris used to send them but they got back to trimble a month ago.

While on the subject I do not understand all of the trimble frame, I've included a file with the raw decoded values of a few RS, what I find most peculiar are the bytes from 35 to 45 (they are numbered at the top) if you look at the line 1244 you can see a change in theses bytes. dec.txt

As I said I have quite a bit of audio recorded from many RS, if you want some I'm willing to share !

rs1729 commented 5 years ago

Maybe one could say Manchester coding is a very short form of code spreading, so considering both raw bits together as 1 symbol is the way to go.

Looks like the first of the 2 bytes before gps-week is the number of satellites, and the 12 bytes after gps-week belong to the (up to) 12 satellites, probably the PRN number (is 32 the upper bound?). [EDIT] Perhaps PRN is only the lower 6 bits, there could be some signal quality bits as well. I have a file with numbers > 32, however the lower bits are disjoint and <=32, looks ok. [/EDIT] Maybe I will check with second GPS-receiver, when I have time. Or https://in-the-sky.org/satmap_worldmap.php

Viproz commented 5 years ago

It does look like GPS satellites in my other recordings too, I don't have anything higher than 32 bits thought.

I took the time to implement the aux decoding and got the data out of all my recordings, here are the files containing the raw data : auxdatYES.txt only correct data auxdat.txt correct data or not

I was expecting something quite simple as auxiliary data but I was apparently mistaken, seems like it's passing a lot of data with different packet format, have you tried working with that already ? I'm going to give it more thoughts tomorrow but I thought I'd share, it's good data all received from one RS launch from METEOMODEM headquarters at Ury (France) on the 2019-01-16 at around 11 am UTC, I do not know what the payload was.

rs1729 commented 5 years ago

I didn't look into the aux data. Once I got a recording from Aire-sur-l'Adour where the first byte was different and the checksums didn't work, when I figured there was actually more data, probably aux-data (e.g. ozone-measurements). But the places I usually receive the M10 don't use it. The aux-apparatus/sensors might send an ID and voltage-measurements as ints or floats that are post-processed later. You can get some ideas from the vaisala-ozone-manual and xdata-descriptions, but the format here are not strings, looks proprietary.

rs1729 commented 5 years ago

There are frames missing in your auxdatYES.txt[OK]. Here I have some consecutive frames from the file with aux I have (probably ozone-sounding): aire28_aux.txt Looks similar to yours, so no special Ury-test-aux.

There is this normal counter-byte (after SN), then the 18 bytes aux. aux: It looks like the data of 1 aux-block is distributed over several frames, 2x5 frames. The 1st byte increases every 10 frames, 2nd byte could indicate the block-part/id, from 0x51 to 0x55, 2 times. Maybe the 2 sub-blocks are for redundancy, looks like it just repeats 2 times for the same 1st byte.

If you have more frames maybe you could see how the data changes. But if it is raw sensor-voltage-readings, it is difficult to translate without more information on calibration.

rs1729 commented 5 years ago

Added more info about GPS data in M10 w/ Trimble GPS. It's the Trimble Copernicus II GPS packet 0x8F-20 (including fix, UTC offset). Looks like the 6449-frames every 10 sec contain signal levels for the tracked satellites (derived from pck 0x47).

rs1729 commented 5 years ago

@Viproz: The Trimble Copernicus II GPS module did it's WNRO last week: https://www.fingers-welt.de/phpBB/viewtopic.php?f=14&t=43&sid=4e4a1d895959aadae404de5a689ef204&start=2950#p269699 https://trimble.app.box.com/s/a43qcip8rarhbfui89lfeq0g8g3sy8e6 Looks like the chip doesn't compare the week with the date in/of the firmware. Don't know if that's the reason why they moved to another GPS and only M10 from 2018/2019 are in the air now. You could handle it in the receiver software I guess, but then again they went back to the old frame data format (for GPS data). Perhaps replacing radiosondes is easier than updating the software at the base station? (After the Vaisala WNRO fix/update in Germany the auto-radiosondes transmitted all on 402.7MHz?! ...)

So the new M10 with AirPrime XM1110 GPS outputs the same frame format as the classic M10 (w/ Trimble). The M10+ w/ G.top GPS was only a short interlude, and as it turns out, Sierra Wireless acquired GlobalTop. (cc: @darksidelemm) For JSON-Output, maybe the extra "-T" is not neccessary?

(JSON: also there is this subtle UTC/GPS thing, DFM transmits UTC, all the others GPS time (don't know about imet-4); however, M10 has the UTC_Offset in the data. Though it's only 18 seconds (2019), not so important.)

ublox handles the WNRO differently: https://portal.u-blox.com/s/question/0D52p00008HKCCpCAP/what-is-the-gps-week-rollover-rollover-and-how-is-it-handled-by-ublox-gnss-receivers Probably the RS41 is save for another 10-15 years, and it could be updated (in principle) before launch.