Support for LCLs (and maybe ECLs) in TMC messages?

[mbirth]

It seems to be easy to get hold of a country's LCL in TMC Exchange format (ISO 14819-3). And in the same package (for Germany) I also got the ECL - but in Excel format which can easily be exported to CSV.

Could you add support to use those files to provide clear text messages for TMC?

[windytan]

Any specifications on TMC Exchange format? Nothing comes up on Google, and there's no mention of it in my copy of ISO 14819-3:2004. Is this in the 2013 revision? If so, where did you get a copy?

The ECL is already hardcoded in English in tmc_data.h. Translation could be possible and would seem to also require language-specific rules for the grammar to work.

[mbirth]

I only found it mentioned in the terms and conditions for the German LCL/ECL - last paragraph. Google found this PDF which looks promising.

About the grammar. The German ECL uses 3 columns: "No quantifier", "Q=1" and "Q>1". And sentences have placeholder(s) for the numbers, e.g.:

"Accident, (L) Stationary traffic" / "An accident, (L) stationary traffic" / "(Q) accidents, (L) stationary traffic"

But there's also a generic all-purpose column reading "(Q) accident(s), (L) stationary traffic".

(L) gets replaced by the length in metres, e.g. "1 km" (Q) gets replaced by the quantity and then there are a few more such codes

There's some more information in that Excel file about extra content which might get sent via additional RDS groups and how that is encoded. You can just order the set from that link above. I got the mail right away, so it seems like nobody checks your data before handing out the precious data.

[windytan]

Thanks, great sources!

[windytan]

LCLs are now supported, experimentally, using the option --loctable "directory_with_loctable".

[mbirth]

It borked for me with:

terminate called after throwing an instance of 'std::runtime_error'
  what():  not supported from ISO 8859-15 (Latin 9) to UTF-8
Aborted

Because the README.DAT was:

1;31/10/2014;;BASt;ISO 8859-15 (Latin 9);2;1

But I don't know how you could make something valid out of that. After changing it to ISO 8859-15, it didn't complain anymore. But I still don't get locations:

{"group":"3A","open_data_app":{"app_name":"RDS-TMC: ALERT-C","oda_group":"8A"},"pi":"0xD321","prog_type":"No PTY","tmc":{"system_info":{"is_encrypted":false,"is_on_alt_freqs":true,"location_table":1,"scope":["urban"]}},"tp":true}
{"group":"8A","pi":"0xD321","prog_type":"No PTY","tmc":{"message":{"description":"Queuing traffic.","direction":"single","event_codes":[108],"extent":"-2","location":34022}},"tp":true}
{"group":"8A","pi":"0xD321","prog_type":"No PTY","tmc":{"message":{"description":"Right lane closed. Roadworks.","direction":"single","event_codes":[501,701],"extent":"-1","location":10257}},"tp":true}

[windytan]

I was a bit too optimistic when I thought the encoding field would ever be a valid encoding name.. :)

Are you loading location table number 1? I should perhaps disable the try/catch statements so that all database parsing errors would crash the program. (I didn't test it on a German dataset myself, I had a Finnish one from 2009.)

[mbirth]

How can I check which tables it loads? I just let it go crazy on the directory with the unpacked LCL data using the new -l parameter. Is there anything else I should check or configure?

[windytan]

It should now print an object called "location_table_info".

[mbirth]

Hmm ... seems there's something wrong (it's printed twice, and judging from the time between the two lines, it's also parsed twice?):

{"location_table_info":{"ltn":1,"num_points":0,"num_roads":0,"num_names":30291}}
{"location_table_info":{"ltn":1,"num_points":0,"num_roads":0,"num_names":30291}}

[windytan]

It's reloaded every time sync is regained or something :) But this will be fixed in the future.

Hmm, seems like points and roads are not succesfully loaded. It could be that column names differ from those in my database. I need to request that German one for sure.

[mbirth]

From what I can see, the column names are correct. Here are the headers and a few lines of data of those files:

NAMES.DAT (column names from locationdb.cc: NID, NAME)

CID;LID;NID;NAME;NCOMMENT
58;1;1;(Basketball-)Arena;
58;1;2;(Fußball-)-Arena;
58;1;3;(Fußball-)Arena;
58;1;4;(Fußball-)Stadion;
58;1;5;48;

ROADS.DAT (LCD, ROADNUMBER, RNID)

CID;TABCD;LCD;CLASS;TCD;STCD;ROADNUMBER;RNID;N1ID;N2ID;POL_LCD;PES_LEV
58;1;44;L;1;1;A448;26551;4409;4389;350;0
58;1;57;L;1;3;L140;;26166;6813;279;0
58;1;82;L;1;4;VG29;;30024;17246;14291;0
58;1;113;L;5;0;;2061;;;14292;0
58;1;118;L;1;3;L170;;25820;20715;576;0

SEGMENTS.DAT (ROA_LCD)

CID;TABCD;LCD;CLASS;TCD;STCD;ROADNUMBER;RNID;N1ID;N2ID;ROA_LCD;SEG_LCD;POL_LCD
58;1;34;L;3;0;B8;;6948;6813;50444;;260
58;1;411;L;3;0;A14;;29429;24782;51281;;268
58;1;1987;L;3;0;L75;;7477;11867;6982;;256
58;1;1990;L;3;0;L75;;11867;18902;6982;;256
58;1;3011;L;3;0;L28;;19352;19223;3009;;529

POINTS.DAT (TABCD, LCD, N1ID, XCOORD, YCOORD, ROA_LCD, SEG_LCD, RNID)

CID;TABCD;LCD;CLASS;TCD;STCD;JUNCTIONNUMBER;RNID;N1ID;N2ID;POL_LCD;OTH_LCD;SEG_LCD;ROA_LCD;INPOS;INNEG;OUTPOS;OUTNEG;PRESENTPOS;PRESENTNEG;DIVERSIONPOS;DIVERSIONNEG;XCOORD;YCOORD;INTERRUPTSROAD;URBAN
58;1;14;P;1;11;;;26528;;3730;;8998;;1;1;1;1;1;1;;;+00684525;+5192775;0;0
58;1;15;P;1;10;;;27953;;5385;;8998;;1;1;1;1;1;1;;;+00684120;+5203450;0;0
58;1;16;P;1;11;;;636;;742;;8998;;1;1;1;1;1;1;;;+00691620;+5213720;0;0
58;1;17;P;1;15;;;10222;;1389;;8998;;0;1;1;0;1;1;;;+00697525;+5219160;24935;0
58;1;18;P;1;11;;;26528;;3730;;9213;;1;0;0;1;1;1;;;+00684525;+5192775;0;0

POFFSETS.DAT (LCD, NEG_OFF_LCD, POS_OFF_LCD)

CID;TABCD;LCD;NEG_OFF_LCD;POS_OFF_LCD
58;1;14;39392;22
58;1;15;22;16
58;1;16;15;17
58;1;17;16;
58;1;18;;26283

[windytan]

The empty fields might have been a problem. I've set default values now.

[mbirth]

Wohooo, that was it!

{"location_table_info":{"ltn":1,"num_points":35765,"num_roads":3820,"num_names":30292}}
{"location_table_info":{"ltn":1,"num_points":35765,"num_roads":3820,"num_names":30292}}

And:

{"group":"8A","pi":"0xD321","prog_type":"No PTY","tmc":{"message":{"coordinates":[[52.36730,13.46670]],"description":"Warning cleared.","direction":"single","event_codes":[1771],"road_number":"B96a"}},"tp":true}
{"group":"8A","pi":"0xD321","prog_type":"No PTY","tmc":{"message":{"coordinates":[[52.42030,13.52240]],"description":"Reopened.","direction":"single","event_codes":[467],"road_number":"A113"}},"tp":true}

Thanks a lot! :smiley:

[mbirth]

Could you make it so that redsea still outputs the location ids, even though it has the location from the table lookup? Same for decrypted locations - it would be nice if it would still spit out the encrypted_location, too.

[windytan]

Sure why not, it should work now.

[mbirth]

You're the best! Thank you very much! :+1:

[windytan]

By the way, I changed the way coordinates are printed, to disambiguate between latitude and longitude.

[windytan]

There doesn't seem to be a standard exchange format for ECLs. Closing this for now.