iMS-100 data availible

[vk-hca]

we have iMS-100 in use in Indonesia, once a week there is one local to here, and daily from another site 200km away. I have recorded some data as I would be keen to get decoding going here. I am using SDR# but I am not sure about audio or baseband format and the bit rate.

Would you like samples of the data?

Up until 2 weeks ago they also would put the processed flight data up on this page a couple of hours after the launch. Hopefully they will resume doing this. http://puslitbang.bmkg.go.id/rason/

regards hca

[rs1729]

If you record FM-audio, don't use any filters, 48kHz sampling rate would be fine, 8bit/sample is ok.

A baseband recording would be interesting, since I have never seen how the signal looks like in frequency space. 30-60 sec at 1 MHz would be nice; maybe I could do a small video example for signal identification. (The audio samples I have are ims-100 and rs-11g I believe, but I'm not absolutely sure which one is which.)

[vk-hca]

Sorry I thought I came back to you before, yes I recorded the data as you described above. Quite a lot of it in fact, but have been busy with other projects of late. I also grabbed the released data for the flight from BMKG.

Can you suggest how to pass it across? I think I could put it up on google drive, files look a bit big for email.

[rs1729]

Yes, something like dropbox or google drive would be fine.

[vk-hca]

I had a look and see I replied about having the data on 5th June to what was probably a no return github email.

Can you mail me on harrycam3 so I can send you the link for the data. I use the normal gmail dot com after that, and in the meantime I will see about uploading the data somewhere for you.

[darksidelemm]

Some iMS-100 samples are here: https://rfhead.net/sondes/meisei/

[darksidelemm]

The audio sample decodes OK with: cat SDRSharp_20190821_004808Z_404404266Hz_AF.wav | ./meisei_ims -2

[rs1729]

Okay, you got some samples. Was going to point to this thread ... Don't have enough data/information, but it could be that the sonde-type or frame-data-version is in the data, for the ims100 samples that I know it is always the same byte at this position. And also there is a 14-16 byte block numbered 0..3 that looks like config data, though sometimes the data in there seems shifted (?), don't know if it is really constant, but maybe the serial number is in there. If someone finds a ims100 that was decoded, it would help. Serial number or ID would be good for sondehub.

FB6230 .... .... 0300: PRNs ?

[rs1729]

After the frame-counter, there is constant data repeating every 64 frame-counts, all the numbers look like not-to-small not-to-big float32. Some are integers, one repeats every 16 frame-counts (smallest period). Probably config/calib-data. The document GRUAN-TD-5_MeiseiRadiosondes_v1_20180221.pdf describes the JMA Format that has some similiarity to the ims100-data (maybe it is the exchange data format?), also including serial number at a similar position in the subframe.

E.g. 2019-06-12:

049DCE 2000 8F31 F386 4AD5 0000 : 4AD5F386 -> 7010755 049DCE 2001 0000 0000 0000 0000 049DCE 2002 800D 8D3A 4AE8 0000 : 4AE88D3A -> 7620253 049DCE 2003 3BE2 0000 4353 0000 : 43530000 -> 211 049DCE 2004 8F31 3054 4ADF 0000 : 4ADF3054 -> 7313450 049DCE 2005 0000 0000 0000 0000 049DCE 2006 8013 3000 470E 0000 : 470E3000 -> 36400 049DCE 2007 3BDA 0000 0000 0000 049DCE 2008 8F31 0000 0000 0000 049DCE 2009 0000 0000 0000 0000 049DCE 200A 800C 0000 0000 0000 049DCE 200B 3BC7 0000 0000 0000 049DCE 200C 8F30 0000 0000 0000 049DCE 200D 0000 0000 0000 0000 049DCE 200E 800F 05E4 4AD9 0000 : 4AD905E4 -> 7111410 049DCE 200F 3BA2 0000 4220 0000 : 42200000 -> 40 049DCE 2010 8F31 F386 4AD5 0000 : 4AD5F386 -> 7010755 ...

There are also numbers with more decimal places, however the one that repeats most often (mod 16) is F386 4AD5 (7010755 as float). Could this be the serial number of the radiosonde? In the recording above from 2019-08-21:

049DCE 3200 8E31 F39E 4AD5 0000 : 4AD5F39E -> 7010767 .. 049DCE 3210 8E31 F39E 4AD5 0000 ..

In this video https://www.youtube.com/watch?v=uP1TiPSxWZM there are 64 parameters, and the Serial Number pops up at FCnt 64: 6015018 (unfortunately audio does not decode).

[rs1729]

2019-08-21, 404.4 MHz (100kHz steps): cal_20190821_404400kHz_64.txt.gz

Frequency might be 15/64: 400000+fq*100 kHz 15 # 0000 4230 : 0x42300000 = 44.000 fq = 404400 kHz

Also matches Lindenberg recordings: 2019-04-02, 402.5MHz: 15 # 0000 41C8 : 0x41c80000 = 25.000 fq = 402500 kHz 2019-04-02, 403.5MHz: 15 # 0000 420C : 0x420c0000 = 35.000 fq = 403500 kHz 2014-10-09, 403.2MHz: 15 # 0000 4200 : 0x42000000 = 32.000 fq = 403200 kHz

[darksidelemm]

I think you mentioned that dft_detect now detects these sondes? If you can get a decoder going with JSON output (even if it doesn't have an ID), I can get a testing branch of auto_rx going so that vk3-hca can at least try and track a few. Recovery might be difficult given where he is is essentially an archipelago, but you never know...

[vk-hca]

Yes Frequency is in the data stream for that JMT data base format we are discussing. 404.4 is the normal DPS freq for all for the recordings I have made, I think one week it was 404.5 but Surabaya is normally 403.6 you might be able to look at the Surabaya sample from a few weeks ago to verify that. Today s Surabaya was on down 404.4 but the modified dft_detect I cut into auto_rx did not pick it up. Quite possibly an antenna related issue. See what happens with the Yagi in the morning if I get to tune it up and put it up high tonight.

Still no update onto days recovered sonde, hoping its still alive on external power with no burst timer, just have to wait.

[darksidelemm]

If it does detect it, auto_rx won't know how to deal with the detection until I add it :-)

[vk-hca]

I cut in the latest version dft_detect and decoder since we have got the dft_detect and IMS100 decoder going and verified it. Cut being quite brutal, much better Mark you do it, as I am a machine code guy who was taught 6800 in hex. Sorry but python is new to me. I see its pedantic about white spaces

[darksidelemm]

Give me some time to figure it out... In the meantime if you want to chat, maybe #highaltitude on Freenode IRC is better than using a github issue log as a chat thread :-P

[vk-hca]

I think this machine wont support, I tried IRC before but it all hung, maybe I have to many anti five eyes and BB blocks installed in my network here. but emails fine, sip voice is better and my normal, are you geared up to run a sip client there? Yes git hubs not the place.

[rs1729]

I think, json-output is already in meisei_ims. But if you wait till tomorrow, I'm including a serial number right now, although I don't know which one is which (until we get some recordings+hardware_pics for confirmation). I think the (main) serial number at frame_count mod 16 that is most frequent and the first number in this cyclic config-data, thus it should be the most appropriate. Since that data is all float32, for aprs the 32bit-machine-representation could be used.

[einergehtnochrein]

After the frame-counter, there is constant data repeating every 64 frame-counts, all the numbers look like not-to-small not-to-big float32. Some are integers, one repeats every 16 frame-counts (smallest period). [...] the one that repeats most often (mod 16) is F386 4AD5 (7010755 as float). Could this be the serial number of the radiosonde? In the recording above from 2019-08-21:

049DCE 3200 8E31 F39E 4AD5 0000 : 4AD5F39E -> 7010767 .. 049DCE 3210 8E31 F39E 4AD5 0000 ..

This is indeed the sonde number as printed on the label. Within the 64 floats #0...#63 it is repeated at positions 0, 16, 32, 48. The other two parameters I've indentified are: 2 = serial number of the PCB 4 = serial number of the sensor boom

Regarding this other finding:

And also there is a 14-16 byte block numbered 0..3 that looks like config data, though sometimes the data in there seems shifted

What exactly is the place in the frame that you are referring to?

[vk-hca]

Yes the meisei_ims decoder JSON output is ok as best I can see for now, I ran a sample DPS sonde wav though it in auto_rx 2 nights ago and and it went through to aprs.fi ok. As I recall I forced it in the c code to the JSON option, also pertaining to that ref the question about wav decode at real time speed and sample data rates.

Re the label serial on the sensor arm, I think the serial full data is on the big label on the polystyrene outer hopefully we will have that in the morning from Eko in Surabaya along with data from the sonde.

[rs1729]

Thanks! I see you have done some further work on Meisei radiosondes, did you have a hardware sample for confirmation?

4 = sn(sensor boom), so this is also a label, but on the sensor boom (below the sensor)? Like this one https://www.gruan.org/gruan/_processed_/2/5/csm_Meisei_IMS-100_01_2f58004c05.jpg ? On pictures, could never read the label on the envelope, but the (main) serial number is on the foam envelope then?!

I think I have seen 4 different sn-like numbers 7xxx in the latest sample: 3200 : 00 # F39E 4AD5 : 0x4ad5f39e = 7010767 3202 : 02 # 8D52 4AE8 : 0x4ae88d52 = 7620265 3204 : 04 # 306C 4ADF : 0x4adf306c = 7313462 320E : 14 # 05E4 4AD9 : 0x4ad905e4 = 7111410 3210 : 16 # F39E 4AD5 : 0x4ad5f39e = 7010767 3220 : 32 # F39E 4AD5 : 0x4ad5f39e = 7010767 3230 : 48 # F39E 4AD5 : 0x4ad5f39e = 7010767

Maybe there are additional or aux sensors possible.

float32 for serial number is unusual, though if all the other config/calib-data for (P)TU is float32, maybe it makes sense to not mix the data types.

FB6230 .... .... 0300: PRNs ?

For ims-100 there is one GPS-block (FB6230) in even frames, and the same block in odd frames contains a subblocks with 12-16 bytes, labeled 0300..0303 (maybe the upper byte can be different) (i.e. a cylce of 4 subblocks). First I thought it would be config data until I saw (in longer recordings) that it is not constant. But it does not change much. And in the 0300-subblock, there are bytes mostly up to 0x20, the last bytes could be C1(C2...) or 00 (older samples). Sometimes there is a byte added or removed from the list. The numbers did match visible PRN-numbers as far as I checked. e.g. ims100_20190612_00Z_404000kHz_raw_cut.txt.gz .... 0300 0103 080B 0E10 1216 171A 1B1F 20C1 C2C3 .... .... 0300 0103 0809 0B0E 1012 1617 1A1B 1F20 C1C2 (09 is added) Although I'm not sure, it would make sense that the odd FB6230-block is also GPS data. Maybe the other blocks contain sat-SNR?

[rs1729]

Included the serial number for --json output (for ims-100 only). For APRS the shorter 32bit-value is maybe more appropriate and recognizable in the raw data. The serial number (ID) may take a few frames until it shows up, similar to DFM.

Frequency drift maybe be as bad as DFM-06/09, also I'm not sure if the baud rate is stable (not good for option -b and error correction). Frequency correction is on the TODO list anyway; basically it works, some more testing.

[einergehtnochrein]

did you have a hardware sample for confirmation?

Yes. Not too long ago there were test launches of iMS-100 from Lindenberg, and most of them went down in Poland. One the Polish hunters was so kind to give me one of his sondes!

4 = sn(sensor boom), so this is also a label, but on the sensor boom (below the sensor)? Like this one https://www.gruan.org/gruan/_processed_/2/5/csm_Meisei_IMS-100_01_2f58004c05.jpg ?

Yes, this is the sensor boom serial number.

On pictures, could never read the label on the envelope, but the (main) serial number is on the foam envelope then?!

Exactly. And there's a third sticker on the bottom side of the PCB.

---

Regarding the GPS data:

The numbers did match visible PRN-numbers as far as I checked.

I did the very same test here before! :-) I'm also convinced that index 03 contains GPS PRN's.

By the way, The byte before the index byte (the upper byte in 0300/0301/0302/0303) seems to be a bit field. I only saw bits 0 and 1 set so far. The GPS position is invalid if either bit is zero. I had the iMS-100 in my car for some "test cruising"... When I drove under a bridge, the sonde would still report a position, but with bit 1 in the flag byte cleared. Bit 0 seems to be always on once the GPS module starts reporting data, which is about 10 seconds after powering up the sonde.

So far I haven't looked much deeper into the GPS fields. I agree with you that both even and odd frames contain GPS data. There is another observation which strongly supports this. The very last 16-bit word in the second block (FB6230) is a checksum for everything GPS related:

There are twelve 16-bit words in each block, lets call them B0[0...11] and B1[0...11]. Take the simple arithmetic sum of these 13 16-bit words: B0[10...11] ... B1[0...10], then B1[11] should be that sum!

This is the same in even and odd frames. As B0[10] and B0[11] contain GPS data at least in the even frames, the existence of the checksum indicates that it's all GPS data waiting to be identified :-)

---

For your information, the GPS module is a SONY brand, and has this marking: SONY D5431-02 812A08G 02

The CPU is a Renesas type RL78/G13

[vk-hca]

Did you have to do some thing to activate the sonde after power down, the ones here from Surabaya apparently don't restart after a power down.

[vk-hca]

The gruan document refers to the chip as "special modified for Meisei", its possibly a variant of the CXD5430 mentioned in this Sony press release. https://www.sony.net/SonyInfo/News/Press/201302/13-022E/

[darksidelemm]

I've added your current meisei decoder and the updated dft_detect into the testing branch of auto_rx. A few notes/questions:

• Currently you emit IMS100-0 when there's no callsign. I'm using this is as a filter, i.e. only accept the telemetry when the number after the - is something other than a 0. This should be fine for now.
• I note that dft_detect returns a negative correlation score on the samples we have, indicating an inverted signal, yet I don't need to use the -i option to decode it using meisei_ims. Are you just auto-detecting the inverted signal?
• I've added a UDP input option to auto_rx. You can use this by running up auto_rx with: python auto_rx.py -v -f 401.0 -m UDP . auto_rx will then be listening on UDP:localhost:50000 for lines of JSON. So you could feed in JSON data using cat samples | ./decoder --json | nc -u localhost 50000. Note that I have disabled all callsign filtering when in UDP mode, so BE CAREFUL! If the uploaders are enabled, then any new callsign you feed in will automatically have a payload document be created on the Habhub tracker, which could lead to bad things if you start feeding in a lot of different callsigns. At the moment APRS uploads won't work with UDP mode, since without knowing the callsign structure I can't produce a shortened callsign.

[einergehtnochrein]

Apparently the sonde can only be restarted if it hasn't been allowed to slowly run out of battery before. If the battery voltage drops below a certain threshold when the sonde is operating, it seems to set a permanent flag which prevents a restart.

My iMS-100 sample was recovered in time to disconnect the battery before that "self destruct" could happen.

This was found out by the Polish guys who managed to collect quite a few of these sondes.

[rs1729]

@einergehtnochrein Thanks for the information! I have seen this checksum in the GPS block, was going to investigate that. The gruan-document said xor-sum, however the document seems to describe a exchange-format that is a bit different.

I got some audio of the last campaign in april. And the first samples in 2015 from the 2014/10 tests at Lindenberg, also with RS-11G. Perhaps the older frame data format (RS-11G) is not relevant anymore, I included a check that tries to separate these two, but I'm not sure. In the non-GPS blocks for ims-100 the two subframes each second have 30C1 or 31C1, resp. https://github.com/rs1729/RS/blob/master/meisei/meisei_ims.c#L48 All ims-100 that I know have C1 in the lower byte. The non-ims100 recordings showed A2 or something else lower than C1 there. Maybe it is a kind of frame/firmware version?

[einergehtnochrein]

Just saw that the C1, C2, C3 in the list of PRN's make sense, as they are used by the Japanese QZSS system. Seems like the availability of QZSS extends from Japan via South East Asia further to Australia.

[rs1729]

@vk3-hca Maybe they use a custom GPS sentence. Meteomodem replaced the M10 GPS chip. The new Sierra Wireless Airprime XM1110 offers custom GPS sentences, and I think they use this for the M10, so the mcu gets a custom GPS sentence with all the GPS information needed in a compact short sentence.

@darksidelemm Yes, the serial number is not in every frame, so (like DFM) until the SN is received, the variable has the old value, at start it is initialized to 0. Would "IMS-XXXX" be better as long as sn==0 ? For APRS I was thinking about the 32bit value that makes the float-SN, but the hex-representation would make 8 characters... Since "IMS-" is 4 characters, the last 5 digits of the serial number might be enough.

The negative score does not matter for Meisei, same as M10 and LMS6-403 (differential coding).

btw: LMS6-403 is 4800 baud and convolutional code. The 1680MHz-Mk2a coding is plain 8N1 9600 baud.

[rs1729]

@einergehtnochrein Beidou has also C-PRN numbers, however would be strange if always C01 is visible, and China is not Japan... So the QZSS data could be the "SBAS" in the product description that @vk3-hca mentioned. The Lindenberg recording shows more 00-bytes, though there is a C1. Maybe from 7000m altitude, from ground the QZS-1 is some 20 deg below horizon.

Lindenberg -> Poland Sometimes there have been photos of Lindenberg radiosondes in a polish forum, but I didn't see Meisei photo. And now the forum has closed the doors I believe ...

[vk-hca]

Auto_rx test on new dft_detect and decoder darksidelemm added into auto_rx yesterday detected and attempted to decode signal just above the noise floor from 250Km distant.

[rs1729]

The signal was detected as meisei, but it didn't decode frames? If the signal is weak, the old decoder is not so good. I have another version that detects the header the same way as dft_detect: https://github.com/rs1729/RS/tree/master/demod/mod Compile: gcc -c demod_mod.c gcc -DINCLUDESTATIC meisei100mod.c demod_mod.o -lm -o meisei100mod

usage: ./meisei100mod --ecc -v (--json optional)

(Hopefully the baud rate is not off)

[darksidelemm]

Will switch auto_rx to using this tonight...

Just to confirm, the IMS100 is transmitting 2400baud AFSK ?

[einergehtnochrein]

It's 2400 baud FSK. (Biphase-S coding, so the bit rate is 1200 bit/s)

[darksidelemm]

Ok Interesting, I was having trouble getting fsk_demod to lock onto it. The spectrum looks quite flat (low modulation index?) Which might be the reason it didn't work...

On Tue., 10 Sep. 2019, 13:02 DF9DQ, notifications@github.com wrote:

It's 2400 baud FSK. (Biphase-S coding, so the bit rate is 1200 bit/s)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rs1729/RS/issues/14?email_source=notifications&email_token=AAH57EYPVCN3T2CD764KN43QI4IMDA5CNFSM4HTMJBU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6JWOKA#issuecomment-529753896, or mute the thread https://github.com/notifications/unsubscribe-auth/AAH57E4EY26GOXJJC5YJMIDQI4IMDANCNFSM4HTMJBUQ .

[einergehtnochrein]

The deviation is more than +/- 3.5 kHz, so with 2400 symbols/s that should be a modulation index of 3 (given that I looked up the right definition). Quite unusual is that there seems to be no baseband filtering at all. A matched filter in the receiver then means no filter...

[vk-hca]

I made those recordings with SDR# on WES7, the filter setting was Blackman-Harris 4, 11.5kc, order1.

[rs1729]

Yes, I calculated a modulation index around h=2.8 or higher. Looks like high BT (not much pulse shaping/filtering).

And the bandwidth is indeed higher than the DFM-bandwidth, but similar parameters, the spectrum looks similar, too.

[rs1729]

@darksidelemm would it be better to output (json) "IMS-XXXX" as long as the ID is not received? https://github.com/rs1729/RS/issues/14#issuecomment-529201555

[darksidelemm]

Yes, this would be better. Given we don't know much about the serial number for at yet that will help me discriminate.

On Tue., 10 Sep. 2019, 15:27 rs1729, notifications@github.com wrote:

@darksidelemm https://github.com/darksidelemm would it be better to output (json) "IMS-XXXX" as long as the ID is not received?

14 (comment)

https://github.com/rs1729/RS/issues/14#issuecomment-529201555

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rs1729/RS/issues/14?email_source=notifications&email_token=AAH57EZGHY3SD7JJ7S7NDIDQI4ZM3A5CNFSM4HTMJBU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6J5P6Y#issuecomment-529782779, or mute the thread https://github.com/notifications/unsubscribe-auth/AAH57E4Y3IVYP4WS2UGBBKLQI4ZM3ANCNFSM4HTMJBUQ .

[darksidelemm]

auto_rx testing branch now uses meisei100mod (and lms6Xmod)

[vk-hca]

WAV and IQ recordings from this mornings DPS sonde, SDR# and WES7 misbehaved worse than normal today, its not as good as it should be. I suspect the QSB is related to that issue.

Filter changed from 12kc to 15kc at 01:00Z

As above by darksidelemm meisei100mod was used today as well, some idea of the good results in screen grabs.

https://drive.google.com/open?id=1vCE6bzrEZBsTMY75HNyxcPo12JMZar0e

[rs1729]

https://tracker.sondehub.org/?sondehub=1#!mt=osm&mz=10&qm=3_days&mc=-8.45444,114.79632&q=RS_IMS*

[rs1729]

I see the upload works. @vk3-hca And I see you uploaded pictures as well.

[rs1729]

I included the gps checksum (c.f. https://github.com/rs1729/RS/issues/14#issuecomment-529186819). For reliable --json output this can be important to check. In the latest upload I found 6/1489 frames where the BCH error correction (and parity bit check) suggested, the frame could be repaired, but the checksum was false. These frames had several BCH-blocks with 2 errors, so if there are blocks with 0 or 1 errors in the same frame, it could be that there were (corrected to) wrong codewords. It is not as likely to get wrong codewords (i.e. not the codeword that was transmitted) as it is for DFM, but the distance between codewords is still not very large, so the checksum is useful, if frames with corrected errors are to be used. If a frame would only be considered when all the codewords had no errors in the frame, the probability for wrong decoding would be very low, but the gain of error correction would be gone. (For DFM a checksum/crc would be really helpfull.)

[darksidelemm]

I've rebased auto_rx's testing branch to your latest meisei100mod. @vk3-hca can you run a git pull, and then re-build the binaries (./build.sh).

[rs1729]

APRS: The serial number has (6-)7 digits now, in hex with 6 nibbles it could up to "16777215" in base 10. Maybe APRS-ID: IMSxxxxxx with 6 hex nibbles is good enough?

[darksidelemm]

Exactly what I've got implemented right now :-) https://github.com/projecthorus/radiosonde_auto_rx/blob/testing/auto_rx/autorx/aprs.py#L85

[vk-hca]

I updated and rebuilt the testing branch after your update to meisei100mod last night for this morning, you have updated since?

[darksidelemm]

Correct, based on the updates @rs1729 pushed early this morning.

[vk-hca]

Ok will do.