Show RadioText for stations that don't transmit end-of-message code

windytan commented 3 years ago

Some stations omit the 0x0D end-of-string character from RadioText. As a result, redsea does not recognize that the whole message was received and never prints the message. The message is only seen when --show-partial is specified. For example, this station is transmitting a 24-character unterminated message:

{"pi":"0x104D","group":"2A","partial_radiotext":"STATUS FM 107.7 STATHMOS","prog_type":"News","tp":false}

This should instead show up in the radiotext field.

Graphical programs like RDS Spy don't have this problem. They don't have to consider whether a RadioText message is complete or not, because they don't produce a decoded log. A complete message will always quickly overwrite a partial one on the screen and the user doesn't notice.

I can think of a few solutions:

Always print every phase of a partial RadioText message by default in the radiotext field (leads to more cluttered output), or
Print partial RadioText as radiotext whenever a new message begins and the previous one didn't have an end code. Checks should be made that no 2A group was missed in between (due to errors). This, too, will randomly display partial messages as complete ones e.g. when the text changes mid-message. For some groups, it will show both fields. Or
Print partial RadioText as radiotext if a message was repeated twice, it wasn't printed for other reasons, and no 2A groups were missed.

With redsea it would be ideal to have each line of JSON only contain self-sufficient information by default (complete messages). But on the other hand, people familiar with RDS are used to seeing strings of text slowly build up from smaller parts.

windytan commented 1 year ago

Elaborating more on the pros and cons of our choices above:

Method 1: Make 'partials' the default

It can be confusing that the default operation of Redsea is to hide incomplete text messages. If the default were changed to display them it would certainly fix this issue. However, it would push the burden of evaluating the completeness of a message to the user, which may not be desirable.

Should we also display partial PS names by default? Or should the existence of the --show-partial option be more clearly communicated?

Method 2: Flush the message buffer when a new message begins

The beginning of a new RT message is signaled with the segment address going back to zero. We could detect this and print whatever is in the message buffer at that point, if the buffer has been contiguously received.

Question: But how can we be sure that the end of the message wasn't lost in noise? Should we somehow track the repeat rate of 2A and detect when a message is probably missed? Or should we just take the risk that some incomplete messages will get printed?

There's an edge case: A new message begins and we have to flush the message buffer. The radiotext field is filled to contain the previous message. But the new message is just 4 characters long and needs to be printed out, too. We can't have two radiotext in the same group. This is probably a highly unlikely scenario, though :)

Method 3: Wait for repeat

If a message repeats identically then we can be somewhat sure that it's a complete message. However, there's an ambiguity in detecting a repeated message with this method; here's an example. Say the message is "CLASSIC FM ". We receive the following 2A groups:

"CLAS"
    "SIC "

The last group is lost in noise. On the second repeat, we get everything:

"CLAS"
    "SIC "
        "FM  "

When the second group is received we detect that it's a repeat of the previous message. But we don't in fact want to print the message yet, otherwise we end up printing "CLASSIC ". Unfortunately there's no way of knowing that there is more text coming.

windytan commented 1 year ago

Regarding Method 2 above, there are stations that will actually forget to transmit the last four characters of the RT message and start the next one already. It would seem like a transmitter bug, however it will only be seen in logging receivers like redsea, and not on actual LCD displays, because it happens so fast.

Take Heart UK, for instance (0xC36C):

Heart - Playing More Music Variety for Norfolk + Nor
                                                ^^^^
Heart - Playing More Music Variety for Norfolk + North S
                                                    ^^^^
Heart - Playing More Music Variety for Norfolk + North Suffo
                                                        ^^^^
Heart - Playing More Music Variety for Norfolk + North Suffo
^^^^ (A/B CLEAR)

If we don't wait for an end-of-message marker we end up printing the string ending in "North Suffo", instead of "North Suffolk". Sometimes, the station will transmit the "lk" along with the end-of-message marker, and no A/B clear flag. On the other hand, this could be useful for debugging RDS encoders.

Solutions brainstorming:

Use method 2 but add a disclaimer-type field to Radiotext output. It could be a boolean flag (is_radiotext_terminated) indicating whether the end marker was received or not.
Use method 2 but if the 0x0D marker is seen, start expecting it for this station from now on.
Wait until 64 characters are contiguously received. Do all stations transmit 64 characters, padded with blanks?

windytan commented 1 year ago

More research in the wiki: https://github.com/windytan/redsea/wiki/Some-RadioText-research

windytan / redsea