search

windytan / redsea

Command-line FM-RDS decoder with JSON output.
MIT License
390 stars 36 forks source link

Block error rate #47

Closed windytan closed 4 years ago

windytan commented 7 years ago

It should be possible to optionally display the block error rate (BLER), i.e. percentage of erroneous blocks during the last n blocks or seconds.

There are two distinct definitions of BLER, depending on the time of calculation: either before (transmission BLER) or after error correction (decoding BLER). Decoding BLER is perhaps more useful for redsea users (?).

The BlockStream member block_has_errors_ is already nearly suitable for this, but for transmission BLER it would need to be able to count error-corrected blocks as well. This would also alter the criteria for sync drop. Whether this is a good thing needs to be tested.

The BLER value should not perhaps refer to a single group; otherwise it could only get one of the values 0, 25, 75, or 100 %. Instead, it should be an average over a dozen or so groups. This may be misleading though, since almost everything else in the JSON object refers to the group at hand.

andimik commented 5 years ago

On one of my PCs I saw by chance that the BER is not a percentage (0-100), but a long number (10 digits or so). I'll check and try to reinstall redsea there ...

windytan commented 5 years ago

Thanks. I haven't seen this behavior, but I'm suspecting this line, where booleans are directly being used in an std::accumulate, will try and change it soon: https://github.com/windytan/redsea/blob/master/src/groups.cc#L140

andimik commented 5 years ago

I found it, it's on my Raspberry 3B. Even after a new compiling it shows that ...

But I have a second installation of redsea thanks to https://github.com/hayguen/fmlist_scan and this works :-)

$ rtl_fm -M fm -l 0 -A std -p 0 -s 171k -g 19.7 -F 0 -f 97.8M | ~/apps/redsea/src/redsea -E Found 1 device(s): 0: Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM Found Rafael Micro R820T tuner Tuner gain set to 19.70 dB. [R82XX] PLL not locked for 1925000 Hz! Bandwidth set to automatic resulted in 290000 Hz. Tuned to 97458000 Hz. Oversampling input by: 8x. Oversampling output by: 1x. Buffer size: 5.99ms Exact sample rate is: 1368000.013046 Hz Sampling at 1368000 S/s. Output at 171000 Hz. Allocating 15 zero-copy buffers {"bler":2147483647,"debug":["TODO 8A"],"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"artificial_head":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"14A","other_network":{"kilohertz":99300,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"compressed":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"debug":["TODO 8A"],"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"dynamic_pty":true},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"group":"14A","other_network":{"kilohertz":88600,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"debug":["TODO 8A"],"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"stereo":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"group":"14A","other_network":{"kilohertz":88800,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"artificial_head":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"debug":["TODO 8A"],"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"compressed":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"14A","other_network":{"kilohertz":99500,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"dynamic_pty":true},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ps":"RADIO-K ","ta":false,"tp":true} {"bler":2147483647,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"debug":["TODO 8A"],"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"stereo":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"3A","open_data_app":{"app_name":"RDS-TMC: ALERT-C","oda_group":"8A"},"pi":"0xA502","prog_type":"No PTY","tmc":{"system_info":{"gap":3,"service_id":0}},"tp":true}

$ rtl_fm -M fm -l 0 -A std -p 0 -s 171k -g 19.7 -F 0 -f 97.8M | redsea -E Found 1 device(s): 0: Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM Found Rafael Micro R820T tuner Tuner gain set to 19.70 dB. [R82XX] PLL not locked for 1925000 Hz! Bandwidth set to automatic resulted in 290000 Hz. Tuned to 97458000 Hz. Oversampling input by: 8x. Oversampling output by: 1x. Buffer size: 5.99ms Exact sample rate is: 1368000.013046 Hz Sampling at 1368000 S/s. Output at 171000 Hz. Allocating 15 zero-copy buffers {"bler":87,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":79,"group":"14A","other_network":{"kilohertz":99900,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":70,"debug":["redsea compiled without TMC support"],"group":"3A","open_data_app":{"app_name":"RDS-TMC: ALERT-C","oda_group":"8A"},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":62,"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":56,"di":{"stereo":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":47,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":39,"group":"14A","other_network":{"kilohertz":99300,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":31,"di":{"artificial_head":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":22,"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":14,"di":{"compressed":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":6,"group":"14A","other_network":{"kilohertz":98600,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2,"di":{"dynamic_pty":true},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ps":"RADIO-K ","ta":false,"tp":true} {"bler":2,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2,"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2,"di":{"stereo":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2,"group":"14A","other_network":{"kilohertz":89100,"pi":"0xA203","tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":0,"di":{"artificial_head":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":0,"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":0,"di":{"compressed":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true}

windytan commented 5 years ago

Ah, I think I found the bug. Just so I remember this, it was introduced in 0.18 when I removed the empty value-initializers, not supported by GCC 4.9, from RunningAverage and RunningSum.

windytan commented 5 years ago

I've pushed a fix to master. Does this work for you?

andimik commented 5 years ago

No, it did not work. Same output.

$ rtl_fm -M fm -l 0 -A std -p 0 -s 171k -g 14 -F 0 -f 97.8M | ./redsea -E Found 1 device(s): 0: NOXON, DAB Stick, SN: 0

Using device 0: Terratec NOXON DAB/DAB+ USB dongle (rev 1) Found Fitipower FC0013 tuner Tuner gain set to 17.90 dB. Bandwidth set to automatic resulted in 0 Hz. Tuned to 97458000 Hz. Oversampling input by: 8x. Oversampling output by: 1x. Buffer size: 5.99ms Exact sample rate is: 1368000.013046 Hz Sampling at 1368000 S/s. Output at 171000 Hz. Allocating 15 zero-copy buffers {"bler":2147483647,"group":"14A","other_network":{"pi":"0xA213","prog_type":"No PTY","ta":true,"tp":false},"pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"stereo":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"di":{"artificial_head":false},"group":"0A","is_music":true,"pi":"0xA502","prog_type":"No PTY","ta":false,"tp":true} {"bler":2147483647,"group":"2A","pi":"0xA502","prog_type":"No PTY","tp":true} {"bler":2147483647,"debug":["TODO 8A"],"group":"8A","pi":"0xA502","prog_type":"No PTY","tp":true}

Have deleted the complete redsea folder and recompiled.

$ ~/apps/redsea/src/redsea --version
redsea 0.18 by OH2EIQ
$ which redsea
/usr/local/bin/redsea
$ redsea --version
redsea 0.17.1 by OH2EIQ

Yes, it has to do with 0.18 as the 0.17.1 works on the same raspberry.

andimik commented 5 years ago 
pi@raspberrypi:~/apps/redsea $ git log
commit c82f1cead2b2f1473f5a3d6c49da3660aa64fee5
Author: Oona Räisänen <oona@kapsi.fi>
Date:   Wed Aug 14 10:01:20 2019 +0300

    fix double-conversion of BLER to percentages
ktb83 commented 4 years ago

I can confirm that this is still not working on a Raspberry Pi 4B. Is there anything I can do to help?

gcc (Raspbian 8.3.0-6+rpi1) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
redsea 0.19-SNAPSHOT by OH2EIQ
windytan commented 4 years ago

Are you seeing this for all signals or a specific one? What does it do instead?

ktb83 commented 4 years ago

It's the same behavior as andimik's output above. I'd be happy to test any suggestions you have.

I'm currently running it like this:

rtl_fm -f 106.3M -M fm -l 0 -A std -p 0 -s 171k -g 42.1 -F 9 -E deemp | redsea -E -t %c -u -e 2>> /run/user/1000/rtl_sdr_fm_player.kxJDbBqd/rds.nvXnW5et | ffplay -nodisp -f s16le -ac 1 -ar 171000 -i pipe:0

Output: redsea_output1.txt

ktb83 commented 4 years ago

gcc warnings 57ce01f054e9278653541f76bd970f20cf911cdd - works OK Output: redsea_output_57ce01f054e9278653541f76bd970f20cf911cdd.txt

integer types, explicit casts - will not compile 2933453071e3f621303576c66ad15fb473557613

float-valued BLER - will not compile 38b71ded50ea8148a0391152e1671558740efc24

minor - will not compile 094a29e33393647a5385a99e7788046605dd80e1

getters for location table values - will not compile d36aa91ce819217467fac8add84ea806d64ac0e7

initializers, etc - Not great, but at least the values are different cf14b406b8114b0841210e701c31d161b91928d1 Output: redsea_output_cf14b406b8114b0841210e701c31d161b91928d1.txt

fine-tune filter bandwidths based on test runs - Not great, but at least the values are different 5086e0decb83d3bbda4974ab5dd2938d4ff7a64e Output: redsea_output_5086e0decb83d3bbda4974ab5dd2938d4ff7a64e.txt

compatibility with GCC 4.9 - Completely broken 9dc21568e4034c4d5ea33dd3891eb756538b3842 Output: redsea_output_9dc21568e4034c4d5ea33dd3891eb756538b3842.txt

ktb83 commented 4 years ago

I tried reverting the changes from: compatibility with GCC 4.9 9dc21568e4034c4d5ea33dd3891eb756538b3842

Unfortunately, it's still broken: Output: redsea_output_9dc21568e4034c4d5ea33dd3891eb756538b3842_2.txt

andimik commented 4 years ago

Could you do a favor and shorten the quoted outputs or even save them into text files and attach them?

windytan commented 4 years ago

Can you test if the current master fixes this?

ktb83 commented 4 years ago

I can confirm that fixes it. Thank you!

{"bler":27,"callsign":"WJQB","group":"2A","pi":"0x6E0D","prog_type":"No PTY","radiotext":"HITS 106","rx_time":"Sun Mar 29 06:50:51 2020","tp":true}
{"bler":25,"callsign":"WJQB","debug":["TODO: Unimplemented ODA app 29555"],"group":"3A","open_data_app":{"app_name":"Enhanced early warning system","message":93,"oda_group":"12A"},"pi":"0x6E0D","prog_type":"No PTY","rx_time":"Sun Mar 29 06:50:51 2020","tp":true}
{"bler":29,"callsign":"WJQB","pi":"0x6E0D","rx_time":"Sun Mar 29 06:50:51 2020"}
{"bler":31,"callsign":"WJQB","pi":"0x6E0D","rx_time":"Sun Mar 29 06:50:51 2020"}
{"bler":31,"callsign":"WJQB","di":{"dynamic_pty":true},"group":"0A","is_music":true,"pi":"0x6E0D","prog_type":"No PTY","rx_time":"Sun Mar 29 06:50:52 2020","ta":false,"tp":true}
{"bler":31,"callsign":"WJQB","group":"2A","pi":"0x6E0D","prog_type":"No PTY","rx_time":"Sun Mar 29 06:50:52 2020","tp":true}
{"bler":33,"callsign":"WJQB","pi":"0x6E0D","rx_time":"Sun Mar 29 06:50:52 2020"}
{"bler":38,"callsign":"WJQB","group":"2A","pi":"0x6E0D","prog_type":"No PTY","radiotext":"HITS 106","rx_time":"Sun Mar 29 06:50:52 2020","tp":true}