projecthorus / radiosonde_auto_rx

Automatically Track Radiosonde Launches using RTLSDR
GNU General Public License v3.0
484 stars 124 forks source link

iMet 50 identified as iMet 54 #567

Closed HoshenKadosh closed 2 years ago

HoshenKadosh commented 3 years ago

Hi, don't know if this is an issue, but I recently recovered what I thought was an iMet 54 as appeared in auto rx, but turned out to be an iMet 50 pilotsonde, without a temperature and humidity probe.

This is the sonde I recovered: 47589938

It seems like temperature and humidity are missing of course in the transmissions as seen in the log file: 2021-08-25T12:04:05Z,IMET54-47589938,43445,31.06025,35.13135,4345.7,-9999.0,-9999.0,-9999.0,-273.0,-1.0,-1.0,IMET5,401.998,17.0,1312,-1,-1.0,-1,-1

So, is there any way, maybe in a certain data in the transmissions that can identify that it is the iMet 50 and by that display that on sondehub? Thanks.

bazjo commented 3 years ago

This is nothing related to your issue. AFAIK, the newer radiosonde models from the south african imet branch are currently in no way documented with regards to hardware. If you have any photos of the inside of these radiosondes, preferably of the PCB with the IC markings visible, that would be greatly appreciated. You can also reach me by email via mail@sondehunt.de

darksidelemm commented 3 years ago

Interesting that it decoded at all! That means that the south african iMet pilotsonde telemetry is almost identical to the iMet-54.

If you can turn it back on and record telemetry that would be useful, it's really the only way we can figure out how to tell these things apart. Also paging @rs1729 .

rs1729 commented 3 years ago

If it still decodes, it's probably a very similar frame structure or the same. There could be a dedicated field for the subtype in the data, or the PTU is just empty, or it is in the serial number. Some sample data would be interesting. Don't know if we discussed this back then, @darksidelemm , the auto_rx sonde type could be just "imet-5" for both, if we knew that the other iMet-branch will not have a different iMet-5 as iMet-4 successor.

HoshenKadosh commented 3 years ago

@rs1729 Here is an IQ sample of the sonde, hope this will help. Download here

darksidelemm commented 3 years ago

Does the serial number that was reported by the decoder match the serial number on the sonde? (if there is any). Some pictures of the inside of the sonde would be interesting - my guess is it's going to look a lot like the iMet-54, just without the sensor stalk connector populated.

HoshenKadosh commented 3 years ago

Does the serial number that was reported by the decoder match the serial number on the sonde? (if there is any). Some pictures of the inside of the sonde would be interesting - my guess is it's going to look a lot like the iMet-54, just without the sensor stalk connector populated.

Yes, the SN does match. Here are some pictures:

IMG_20210827_162558.jpg

IMG_20210827_162549.jpg

IMG_20210825_232836.jpg

darksidelemm commented 3 years ago

As expected, same PCB with some components not populated. Compare with a iMet-54 here: https://twitter.com/vk5qi/status/1429720345301323781

darksidelemm commented 3 years ago

Some example raw frames from the iMet-50 recording:

02D62A320747B56001E06694020CB5D2000001D000000000000000004E6E6B284E6E6B284E6E6B28000D00300000000000000000000000000000000000000000000000000000000000000000000000000000F80000000000770000005AD2051053CC50D1FE9711154000EBE9
02D62A320747B94801E06695020CB5D2000001D100000000000000004E6E6B284E6E6B284E6E6B28000D00300000000000000000000000000000000000000000000000000000000000000000000000000000F8000000000077000000213106100009141F1C71404D4000466D
02D62A320747BD3001E06695020CB5D2000001D100000000000000004E6E6B284E6E6B284E6E6B28000C00300000000000000000000000000000000000000000000000000000000000000000000000000000F800000000007700000040420710404E5D458CE57F0140006B0A
02D62A320747C11801E06695020CB5D2000001D100000000000000004E6E6B284E6E6B284E6E6B28000C00300000000000000000000000000000000000000000000000000000000000000000000000000000F80000000000770000007FC208100000000071FE00004000FCA5
02D62A320747C50001E06695020CB5D2000001D200000000000000004E6E6B284E6E6B284E6E6B28000C00300000000000000000000000000000000000000000000000000000000000000000000000000000F800000000007700000000000910000000004B8300004000B060
02D62A320747C8E801E06695020CB5D2000001D200000000000000004E6E6B284E6E6B284E6E6B28000C00300000000000000000000000000000000000000000000000000000000000000000000000000000F800000000007700000000000A10008F0000B7DF0D174000C761
02D62A320747CCD001E06695020CB5D2000001D200000000000000004E6E6B284E6E6B284E6E6B28000C00300000000000000000000000000000000000000000000000000000000000000000000000000000F800000000007700000081030010000008580DD900004000D0C2
02D62A320747D0B801E06695020CB5D2000001D200000000000000004E6E6B284E6E6B284E6E6B28000C00300000000000000000000000000000000000000000000000000000000000000000000000000000F80000000000770000000600011005FF7530A62710084000860F

.. and from an iMet-54:

034835FE0E0E05C0FE7A431E01AE4FDA00003B4A00000000000000004184C41742C80000C23CB2145C08003EB40073F06800093F000008355C0002CF1300084D120000001600000019000000A40000009B00F806C3A0000000000042060A010000FF738CC2EE075B400016F9
034835FE0E0E09A8FE7A431E01AE4FDA00003B4A00000000000000004184B966000000004E6E6B2819080032F90070F1D300090211000835A30002CFFE00084D01FFFFFFFE000000FFFFFFFFA4FFFFFF970F3206C3C380000000044202560200AC1006A4BA6BAE1F4000EE19
034835FE0E0E0D90FE7A431E01AE4FDA00003B4A00000000000000004184B5B442C578E741D5D5009008003E2C0070F15F000903F10008351B0002CE1000084D15000000140000001C000000A40000009700F806C3CC800000000042A4160300AD1AA91B7B4AAD1D4000A248
034835FE0E0E1178FE7A431E01AE4FDA00003B4A00000000000000004184A9A642C80000C264206C7608003E3F0074B02C0009501C000836D30002CF0100084D010000000100000001000000A40000009C0F3206C3E6800000000442AB20040026289E03691415044000A804
034835FE0E0E1560FE7A431D01AE4FD900003B4A00000000000000004184ADFD000000004E6E6B2816080032FE006F73EB0008E419000835D30002CFFF00084DFEFFFFFF01FFFFFF02000000A4000000950F3206C3AE80000000044200420500004C004A6FBB00534000B31B
034835FE0E0E1948FE7A431D01AE4FD900003B4A00000000000000004184BD18000000004E6E6B283308003288007272F50009211D000835DD0002CF0700084D14000000140000001F000000A40000009900F806C3B100000000004200460600550000436BE755004000FE79
darksidelemm commented 3 years ago

I'm wondering if the presence / lack of PTU data could be used? I see a lot more zeros in the iMet-50 frame.

rs1729 commented 3 years ago

Yes, there is a large zero block after the two bytes 42, 43 that are some kind of status bytes. (byte 42 is 00 and) byte 43 is 0x3E for iMet-54 if everything is OK. I believe the upper nibble is GPS and the lower could be PTU. (Right now only the upper nibble is checked.) The bytes after the status could be raw PTU sensor values, have seen frames with FFFFFF there and status 0032. For iMet-50 it is all zero and status 0030.

The PTU data that is used in the decoder is between the GPS position data and the status bytes, this is pre-calculated temperature and humidity and sensor rh-sensor temperature as float32. For iMet-50 all the float32 values in the sample recording are 0x4E6E6B28 : 1000000000.0000. I have seen this value also in iMet-54 frames with status 0032, the rh Temp, perhaps broken sensor. Could be the default error value.

Maybe there is a byte that indicates the subtype or the sensor setup.

The large zero block after the status byte could be a good indicator for iMet-50, maybe together with all PTU = 0x4E6E6B28 : 1000000000.0000. I hope this cannot occur in single iMet-54 frames.

The JSON "type" is already IMET5. Now there are different possibilities. The prefix to the serial number could be either IMET5-, or IMET54-/IMET50- depending on the subtype. And there could be a "subtype" IMET54/IMET50 as for other radiosondes.

@HoshenKadosh do you have recordings from the flight, or maybe next time you see it in the air?

darksidelemm commented 3 years ago

Hopefully the type can be identified in one frame. That way your second option (IMET54- / IMET50-) is doable. Adding in a subtype field is also a good idea, and will flow well to the Sondehub db. I think I might start referring to these as iMet-5x sondes. They are the same hardware after all!

rs1729 commented 3 years ago

I assume that this zero block is zero all the time for iMet-50, and probably there is always data in there even at start-up for iMet-54. To put it into the SN is more critical in case it changes in a single frame or the detection fails. I also have to check the minimal number of bytes such that the frame is accepted (all the interesting data is before the status byte). So something like iMet-50:

{ "type": "IMET5", "frame": 44561, "id": "IMET5-47589938", ..., "subtype": "IMET50" }

and iMet-54:

{ "type": "IMET5", "frame": 42383, "id": "IMET5-55064506", ..., "subtype": "IMET54" }

would not produce multiple IDs in case of false type detection. (Just hope that the other InterMet branch with the iMet-4 does not come up with a totally different iMet-5 ...) So there might be a problem when different auto_rx versions upload there IDs...

Don't know if the serial number ranges are separated like it seems to be for LMS6-403 and LMS6-1680. Would be easiest to put a dedicated type byte in the data. At least it is not written in ASCII as for RS41, as far as I can see.

HoshenKadosh commented 3 years ago

Yes, there is a large zero block after the two bytes 42, 43 that are some kind of status bytes. (byte 42 is 00 and) byte 43 is 0x3E for iMet-54 if everything is OK. I believe the upper nibble is GPS and the lower could be PTU. (Right now only the upper nibble is checked.) The bytes after the status could be raw PTU sensor values, have seen frames with FFFFFF there and status 0032. For iMet-50 it is all zero and status 0030.

The PTU data that is used in the decoder is between the GPS position data and the status bytes, this is pre-calculated temperature and humidity and sensor rh-sensor temperature as float32. For iMet-50 all the float32 values in the sample recording are 0x4E6E6B28 : 1000000000.0000. I have seen this value also in iMet-54 frames with status 0032, the rh Temp, perhaps broken sensor. Could be the default error value.

Maybe there is a byte that indicates the subtype or the sensor setup.

The large zero block after the status byte could be a good indicator for iMet-50, maybe together with all PTU = 0x4E6E6B28 : 1000000000.0000. I hope this cannot occur in single iMet-54 frames.

The JSON "type" is already IMET5. Now there are different possibilities. The prefix to the serial number could be either IMET5-, or IMET54-/IMET50- depending on the subtype. And there could be a "subtype" IMET54/IMET50 as for other radiosondes.

@HoshenKadosh do you have recordings from the flight, or maybe next time you see it in the air?

I don't have more recordings, but I can record in the next launch if needed.

darksidelemm commented 3 years ago

You could also turn on this option: https://github.com/projecthorus/radiosonde_auto_rx/blob/master/auto_rx/station.cfg.example#L408

That'll save the required data into the log directory.

darksidelemm commented 3 years ago

Out of curiosity, I had a go at removing the sensor stalk from the iMet-54 I have on hand here, and powered it up. This resulted in the following raw data:

00BC614E0041C3F0FDF29A4908400E22000002D000000000000000004E6E6B28000000004E6E6B28000807300000000000000000000000000000000000000000000000000000000000000000000000000000F80000000000770000005500080055005500AF7D55004000469F
00BC614E0041C7D8FDF29A4908400E21000002D300000000000000004E6E6B28000000004E6E6B28000A07300000000000000000000000000000000000000000000000000000000000000000000000000000F80000000000770000005500090055005500B32255004000ED62
00BC614E0041CBC0FDF29A4908400E21000002D300000000000000004E6E6B28000000004E6E6B28000A07300000000000000000000000000000000000000000000000000000000000000000000000000000F800000000007700000055000A0000265500B828000E4000745F
00BC614E0041CFA8FDF29A4908400E21000002D500000000000000004E6E6B28000000004E6E6B28000A07300000000000000000000000000000000000000000000000000000000000000000000000000000F80000000000770000008101000000060860D8AB01004000D1C2

Note that the serial number reported was '12345678', suggesting that the iMet-54 sends the sensor stalk serial number. Hopefully there are some other differences that might tell the difference between an iMet-54 with no sensor stalk, and an iMet-50?

Putting the stalk back on results in this:

0348422F00457540FDF29A3F08400E2A000003C60000000000000000419A7298424C252941A316507605003EDF0075EC4F00084AEC00081F050002CCB60008480A0001B0CA000227100001B0660002279D00F8084D540000000000425500090055005500913555004000B3E5
0348422F00457928FDF29A4108400E2B000003C80000000000000000419A3ECD424C325641A29C30F405003EDE0072EB530008160300081FFD0002CDC3000847000001B0D7000227050001B0660002279900F8084D6400000000004255000A00002655004B0C000940004435
0348422F00457D10FDF29A4108400E2B000003CC0000000000000000419A13B2424C698041A26B40B805003E3300752C91000841FD00081F1E0002CCCF000848FB0001B0D1000226FC0001B0660002269C00F8084D9C0000000000428101000000000860680C0100400098A8

This decoded to serial number 55067183, which matches what is written on the stalk itself. On the main PCB is another serial number (47590359), which I guess is what is transmitted for the iMet-50. This doesn't appear in the telemetry.

rs1729 commented 3 years ago

The frames without the sensors look very similar to iMet-50, but the "status" bytes are 0730, the first byte is not zero but 07. Also [T 4E6E6B28 : 1000000000.0000] [RH 00000000 : 0.0000] [Trh 4E6E6B28 : 1000000000.0000] where the real iMet-50 has all 4E6E6B28 : 1000000000.0000. Probably the non-zero first byte in 0730 indicates, that some hardware is missing. (The byte before that could be GPS satellites?)

The serial number is the first 4 bytes: 0x00BC614E=12345678. Didn't see anything like that for the PCB-SN. Not enough frames here, but there is a cycle of 11 blocks at the end with a counter 0x00..0x0A in the middle, but the data is not really constant, maybe parts are constant. EDIT: The data of the real iMet-50 shows the serial number that is also on the PCB, so it knows there is only this number.

Can you make a recording when the radiosonde powers on? The very first frames?

HoshenKadosh commented 3 years ago

Can you make a recording when the radiosonde powers on? The very first frames?

@rs1729, Yes, download link Sorry for the delay.

rs1729 commented 3 years ago

No problem, thanks! Actually I was responding to @darksidelemm , but it does not matter, I guess. Power-on should be similar for all imet5. Re: https://github.com/projecthorus/radiosonde_auto_rx/issues/567#issuecomment-907621537 @darksidelemm where (in JSON) would you prefer the distinction imet54/imet50?

darksidelemm commented 3 years ago

Keep the "type" field as IMET5, but add in a "subtype" field with either "iMet-54" or "iMet-50". That should work nicely!

rs1729 commented 3 years ago

@darksidelemm Yes, but what about the serial number prefix?

iMet-50: { "type": "IMET5", "frame": 44561, "id": "IMET50-47589938", ..., "subtype": "IMET50" } or alternatively { "type": "IMET5", "frame": 44561, "id": "IMET5-47589938", ..., "subtype": "IMET50" } ? (same for iMet-54)

Now we have iMet-54: { "type": "IMET5", "frame": 42383, "id": "IMET54-55064506", ... }

a) If I add the imet50-check, i.e. if TU sensor data is empty etc., not sure how reliable it would be, i.e. if it could happen that an iMet-54 would be identified as iMet-50 for some frames if the sensors break down (e.g. after burst) or something similar. Then the ID would change.

b) On the other hand, if we changed the prefix to IMET5-4xxx for both, then older versions of auto_rx would still use IMET54-4xxx producing multiple IDs on Sondehub.

darksidelemm commented 3 years ago

The following will work:

If only subtype changes with the detection of iMet-50/iMet-54 that will have limited impact if it is detected incorrectly - you won't end up with multiple sondes, you'll just see the reported subtype of the sonde flip between iMet-54 and iMet-50, which isn't a huge issue.

rs1729 commented 3 years ago
(the prefix is only used within auto_rx to name a log file).

Okay, then this not a problem. Added subtypes IMET54 and IMET50, e.g.

{ "type": "IMET5", "frame": 44560, "id": "IMET5-47589938", "datetime": "...Z", "lat": ..., "lon": ..., "alt": ..., "subtype": "IMET50" }

(Could RS92/RS41 or M10/M20 produce collisions?)

TheSkorm commented 3 years ago

sub type and type is always included in the data that we save so if a collision does occur the backend will/should be fine. Currently our frontend assumes there won't be any collisions and we haven't observed one yet so we haven't worked on that problem yet.

darksidelemm commented 3 years ago

I've added this into the testing branch. @HoshenKadosh if you could give it a go that would be appreciated - not sure how often you see iMet-50 sondes though!

HoshenKadosh commented 2 years ago

Sadly since then i haven't seen more launches of the iMet 50. The launch site doesn't have a schedule as I have seen, and only launched a few iMet-50 sondes. However, i turned on the sonde I recovered, and it seemed to be detected as an iMet-50, so I think this can be closed. Thanks.

rs1729 commented 2 years ago

Added CRC check. The bit order made it more difficult to analyze, but seems to work.