Message 24 fails to decode

[nebukadnezar]

These two examples of message type 24 decode successfully using maritec's decoder, but fail with libais with a decoding error:

!AIVDM,1,1,,A,H7OfcLPt<D4r1L5HD0000000000,0*01 !AIVDM,1,1,,A,H7Ofom0t<D4r0HiTE8000000000,0*00

[jamtho]

Both decode correctly using padding bits value of 2, which I think is the correct value? (Giving message length 160.)

From http://catb.org/gpsd/AIVDM.html#_type_24_static_data_report:

"According to the standard, both the A and B parts are supposed to be 168 bits. However, in the wild, A parts are often transmitted with only 160 bits, omitting the spare 7 bits at the end. Implementers should be permissive about this."

Where did the messages come from?

[nebukadnezar]

I'm decoding NMEA strings received/demodulated using rtl_ais. Indeed with correct padding it works. I ran it through the stream decoder as well, which I thought did its own padding, and it fails there as well.

[jamtho]

As far as I know, libais always trusts the nmea padding value. The checksums are right too, I think (I checked the first anyway), so the source genuinely is producing messages with 2 bits of spare at end. Annoying!

It's up to Kurt whether he wants to support such messages.

Are you seeing a lot of them, out of interest? The sources I personally use filter them out, I'm pretty sure, so I've no idea how common they are.

Presumably the workaround is to manually change the padding value to 2 when you get a message of type 24, with that string length and broadcast padding 0. But it's not exactly nice. You could always have a separate "recover broken message" path in the control flown that implemented any such rules like this when the decode failed, but obviously it depends on your use case.

[nebukadnezar]

Interesting - I can address that special case individually based on message length. Any idea what's happening with these guys here? They don't decode regardless of padding bits. (having trouble inserting them because they have markup for code characters in them - remove one of the adjoining single quotes mid sentence)

!AIVDM,1,1,,B,37P3aB0001:konad``S2``AHK200SQ0,0*16 !AIVDM,1,1,,A,?7QQiG0k00:lDOid``PU10@0,0*1F !AIVDM,1,1,,B,37P1iG0PAs:lCOId``IqRNAr00Q610,0*5A

The middle one doesn't decode with maritec either.

And the answer your other question about their frequency: Out of about 250000 messages, only 4 were of the wrongly padded type.

[jamtho]

For messages (1) and (3) remove the zero at the end of the body and they decode fine. Spotted because the body length was exactly one char (six bits) too long (and it was a zero).

I'd actually be interested in another project to catalogue and possibly fix up systematically broken AIS messages like these; there may be one already. Not sure.

Message (2) is more difficult. There is a fixed set of valid message sizes here, see: https://github.com/schwehr/libais/blob/4e9b6d6ae4ee02ed2bdb4a9d7835cff4780287d5/src/libais/ais15.cpp#L15-L19 and http://catb.org/gpsd/AIVDM.html#_type_15_interrogation (below table) This doesn't hit any of them. You can get some kind of decode if you trim to those bounds, but I very much doubt it's the real contents.

[nebukadnezar]

Thanks! I was suspecting that also, but haven't seen it really except for in these few rare messages. Do you come across mysterious messages like this frequently?

Happy to share raw NMEA output from my recordings of that's of any help with your catalogue.

[jamtho]

No problem! Unfortunately almost all the data I get - including the terrestrial stuff - comes via the satellite providers, who like to filter out the weird stuff to avoid confusing their customers. This debugging stuff is more of a hobby for me :)

It would honestly be helpful to set up a large general store for AIS data that everyone could use; I'm thinking maybe a public BigQuery table or similar, for pre- and post-decoded data. I know lots of academic types that would appreciate it.

The most obvious data source would be AisHub, but naturally I don't live close enough to the sea to pick up any data to share :). If you had access for a while and wanted to bung a load of data at me I'd gladly do the honours!

[nebukadnezar]

Looks like AisHub would do what you need, no? Or do they sanitise the data as well?

[jamtho]

AisHub would be ideal, but you need to share your own AIS feed to join, and since I'm nowhere near the sea I'm looking for someone else's to piggyback on. I'll get there in the end, there's no hurry.

[nebukadnezar]

Ah... now that makes sense... email me to discuss options. No idea how to transmit my email to you, so using this single use link thing I got: https://secure.inside.net/SimplySecure/get.php?token=ED0D087A-7967-8FF8-B702-0211FA757F59