windytan / redsea

Command-line FM-RDS decoder with JSON output.
MIT License
390 stars 36 forks source link

TMC from TomTom #61

Closed veso266 closed 5 years ago

veso266 commented 5 years ago

Hi I was looking for TMC Location tables for TrafficNav as TrafficNav would not send them to me, I though that I could probably get them from a Navigation

I chose TomTom as I knew they have an agreament with TrafficNav so I downloaded a map for a recent TomTom and in there were tmctables.ttd (which I assumed that that had to be TMC Locataion Tables), but I don't know if they are an apropriate format for RedSea to read

I uploaded them here: http://pskk.si/oldWP/vid/TMC Location Tables.rar

so can you just check if they are in apropriate format? (I didn't find much on TomTom .dat file format) there were (in tmctables.ttd) some clear strings that could be areas but I do not know

found something for ttbin format: https://github.com/FluffyKaon/TomTom-ttbin-file-format but nothing for ttd

Thanks for Anwsering and Best Regards

windytan commented 5 years ago

These don't appear to be in the format required by Redsea (TMC Exchange), but instead in some kind of compressed binary format.

The TMC Exchange format is in ASCII. I found a description here: http://www.servotechnica.spb.ru/library/BOOKS/GPS/ExchangeFormat.pdf

veso266 commented 5 years ago

Compressed binary? (but why do I see clear text in there) I believe that its some kind of collection for all location tables from ciries that map covers (Austria, Italy, Hungary, Slovenia, etc)

at Offset: 964321 (decimal): cities and road sections from Austria start this clear text then continues to offset: 1005813 (decimal) where some wierd data start again (for some reason it always starts with 00 18 (HEX), some kind of header?, but always with 00 18 00 if it starts when clear text data ends)

also on a side note: I am trying to get TMC Provider Name, but am unable to do so RedSea should parse that but for some reason it doesn't here is the test file I was trying this on (it shuold say Traf_Nav): http://www.pskk.si/oldWP/vid/933A%20-%202018-09-23%2013-32-59.spy

RDS Surveyor parsed that as Traff but it took a long time for some reason

but on Italien public TMC Service carried on RAI Radio 3 even RDS Surveyor was not able to parse TMC provider name: http://pskk.si/oldWP/vid/5203%20-%202018-09-23%2013-06-23.spy (not sure if stations are supposed to broadcast that?)

PS: do you have road and snow icons you used here: http://3.bp.blogspot.com/-f908_RqveDw/UYU31DhtrBI/AAAAAAAAB5o/NqpaVfCakeM/s1600/Screenshot-1.png somewhere as they realy look great and would like to have them that also happend on our public TMC carried by Radio Slovenija 1: pskk.si/oldWP/vid/9201 - 2018-09-23 13-34-35.spy no TMC Provider Name

mvglasow commented 5 years ago

but on Italien public TMC Service carried on RAI Radio 3 even RDS Surveyor was not able to parse TMC provider name: http://pskk.si/oldWP/vid/5203%20-%202018-09-23%2013-06-23.spy (not sure if stations are supposed to broadcast that?)

Seeing the same here (with RDS Surveyor and Qz, which is based on it), presumably RAI does not broadcast a service name. All three stations carry the same TMC service (same CC/LTN/SID). My commercial sat-nav (Audi RNS-E) will display the station name instead of a service name.

mvglasow commented 5 years ago

As for Slovenia 1 (9201), I see the same with RDS Surveyor. This is also congruent with another sample I have. The station may not broadcast a service name at all, or not in the time period of the sample (where service names are broadcast, this happens with a low frequency).

On 933A I see RDS Surveyor parses that as Traf????, i.e. only the first half is decoded. (The identifier is transmitted in two halves, so you might have picked up only the first.)

veso266 commented 5 years ago

[quote]Seeing the same here (with RDS Surveyor and Qz[/quote] Qz any link? I never heard it existed

mvglasow commented 5 years ago

https://gitlab.com/mvglasow/qz, a TMC app for Android, relies on RDS Surveyor for RDS/TMC decoding

andimik commented 5 years ago

Thanks for pointing this out, I even did not know such a provider name is sent within RDS.

I tried with Radio SI (I am living very close to Slovenia), and I get the provider, even in RDS Surveyor

image (see screenshot)

$ cat 933A_-_2018-03-08_17-40-20.spy | redsea -h -p | grep prov {"group":"8A","pi":"0x933A","prog_type":"Travel","tmc":{"service_provider":"Traf_Nav"},"tp":true}

http://andimik.bplaced.net/rds/spyfiles/933A_-_2018-03-08_17-40-20.spy

But i seems on PRVI (9201, 1st public channel in Slovenia) the provider name is not transmitted.

I can receive RAI 1-3 and Isoradio, but they don't have TMC in Northern Friuli. On the 4th Minority program (Trst A) the provider name is not present (have samples for many years).

In Austria it's called OE3

$ cat A201_-_2018-12-09_21-29-43.spy | redsea -h -p | grep prov {"group":"8A","pi":"0xA201","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":false} http://andimik.bplaced.net/rds/spyfiles/A201_-_2018-12-09_21-29-43.spy

At RTL102.5 it's TMC.IT

$ cat 5218_-_2015-03-28_10-27-25.spy | redsea -h -p | grep prov {"group":"8A","pi":"0x5218","prog_type":"Varied","tmc":{"service_provider":"TMC.IT "},"tp":true} {"group":"8A","pi":"0x5218","prog_type":"Varied","tmc":{"service_provider":"TMC.IT "},"tp":true} http://andimik.bplaced.net/rds/spyfiles/5218_-_2015-03-28_10-27-25.spy

andimik commented 5 years ago

Update: I found a very cool solution how to search within hundreds or even thousands of sample files I took over the past years:

Assuming no file is named ----* then do this:

$ cat *.spy >> /tmp/all_spy_files.spy

Then you can search in the archive and only display unique lines (takes a while to sort)

$ cat /tmp/alt_spy.spy | redsea -h -p | grep prov | sort | uniq {"group":"8A","pi":"0x232F","prog_type":"Information","tmc":"service_provider":"CRO_A_GA"},"tp":true} {"group":"8A","pi":"0x5218","prog_type":"Varied","tmc":{"service_provider":"TMC.IT "},"tp":true} {"group":"8A","pi":"0x9201","prog_type":"No PTY","tmc":{"service_provider":"Traf_Nav"},"tp":false} {"group":"8A","pi":"0x9202","prog_type":"No PTY","tmc":{"service_provider":"Traf_Nav"},"tp":true} {"group":"8A","pi":"0x9203","prog_type":"No PTY","tmc":{"service_provider":"Traf_Nav"},"tp":false} {"group":"8A","pi":"0x933A","prog_type":"No PTY","tmc":{"service_provider":"Traf_Nav"},"tp":true} {"group":"8A","pi":"0xA201","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":false} {"group":"8A","pi":"0xA203","prog_type":"News","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xA203","prog_type":"Pop music","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xA213","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":false} {"group":"8A","pi":"0xA404","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xA502","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xA602","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xA707","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xA802","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xA902","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xAA0A","prog_type":"No PTY","tmc":{"service_provider":"OE3 "},"tp":true} {"group":"8A","pi":"0xB201","prog_type":"Other music","tmc": "service_provider":"Traf_Nav"},"tp":true} {"group":"8A","pi":"0xB206","prog_type":"Pop music","tmc":{"service_provider":"TraffNav"},"tp":true} {"group":"8A","pi":"0xC202","prog_type":"No PTY","tmc":{"service_provider":"Traf_Nav"},"tp":true}

So you will at least get the station with the grepped line, but not the filename itself ...

This is my complete archive:

aktuelle_spy.spy.zip alt_spy.spy.zip

Have fun!!!!!

windytan commented 5 years ago

You could use this bash magic to find out the file names (though the output is quite verbose):

for FILE in *.spy; do echo "File: $FILE"; redsea -h < $FILE | grep prov | sort | uniq; done

About the traffic sign icons. Sorry for the late reply, but I think I made these icons myself based on SVG images from Wikimedia Commons. I added a subtle gradient in Inkscape and rendered them as small PNG images. The ones I made for my project in 2011 are here.

mvglasow commented 3 years ago

@veso266 seems you have a bunch of samples, would you mind adding them to https://github.com/walczakp/rds-spy-logs/ ?

@windytan this repo might be of interest to you as well, there’s a collection of samples from various countries which make for a nice collection of test cases and are otherwise quite difficult to get hold of

andimik commented 1 year ago

Regarding

for FILE in *.spy; do echo "File: $FILE"; redsea -h < $FILE | grep prov | sort | uniq; done

How does the code look like when there is a space in the filename?

Of course I could replace the Space by _ by

for file in *; do mv "$file"echo $file | tr ' ' '_'; done

windytan commented 1 year ago

Try to add quotation marks - redsea -h < "$FILE"