GA aircraft appear as Airbus or Boeing airliners

[172MC]

On a clean installation of XP 11.26, with LiveTraffic 0.81 and Bluebell CSLs + required xsquawkbox files, I tested this morning at my "home" airport Centennial (KAPA), comparing to the traffic that appears on Flightaware. LiveTraffic shows the correct number of nearby planes, the correct registration number and their approximate locations, but NOT the correct type. KAPA is essentially a GA airport but LiveTraffic only displays Airbus and Boeing airliners. For example, the screenshot attached shows an airliner with registration N5306U, which on Flightaware is identified as a C172. The A/C information window when this registration number is entered has a blank field for the a/c type (sorry forgot to take a screen shot of the A/C info window).

Log.txt

[172MC]

After testing the clean XP11.26 installation, I tested my normal, rather complex 11.26 version, again at KAPA. LT loaded nicely and showed the correct number of aircraft, correct registrations and approximate positions compared to Flightaware, but, once again, wrong aircraft type (airliner rather than GA). This time I took a screenshot of the A/C info window. Here it is along with the log.txt

Log.txt

[172MC]

Added ADS-B Exchange and now many aircraft show correct type (and in total, a lot more aircraft)! I was surprised that I can have both OpenSky and ADS-B Exchange checked at the same time.

Log.txt

[TwinFan]

A320 is the "default" a/c type if the LT/multiplayer library can't decide anything better. Here LT probably couldn't get hold of good a/c master data. In OpenSky the pure tracking data contains no a/c info. That's why there is the additional masterdata channel. If LT only receives tracking data but cannot get master data then it's gonna be an A320.

For the second case Log analysis was easier as the transponder code A2AF88 is there. And the log says about that plane:

LiveTraffic 1543863253.4 DEBUG src\LTChannel.cpp:316/FetchAllData: OpenSky Masterdata Online: Sending HTTP: https://opensky-network.org/api/metadata/aircraft/icao/A2AF88
LiveTraffic 1543863253.8 DEBUG src\LTChannel.cpp:332/FetchAllData: OpenSky Masterdata Online: Received 621 characters
...
LiveTraffic 1543863254.0 WARN  src\LTChannel.cpp:1294/ProcessFetchedData: OpenSky Masterdata Online: Ignoring invalid data for 'A6B3AA'

So LT couldn't process that master data. However...621 received chars looks about correct. Sending that query again now, days later, my browser receives a totally valid answer of, chacka, 621 characters:

{"registration":"N272SP","manufacturerName":"Cessna","manufacturerIcao":"","model":"172S","typecode":"","serialNumber":"172S8016","lineNumber":"","icaoAircraftClass":"","selCal":"","operator":"","operatorCallsign":"","operatorIcao":"","operatorIata":"","owner":"Blue Ryan Llc","categoryDescription":"Light (< 15500 lbs)","registered":null,"regUntil":"2020-09-30","status":"","built":"1998-01-01","firstFlightDate":null,"engines":"LYCOMING I0360 SER A&C","modes":false,"adsb":false,"acars":false,"vdl":false,"notes":"","country":"United States","lastSeen":null,"firstSeen":null,"icao24":"a2af88","timestamp":1527559200000}

That would mean that LT failed processing some of that data and bailed out. That would be a bug...and the other a/c next to the A2AF88 in the log, which is A6CC0C, is probably the same case. Again, now I do receive good looking data.

Just an idea to be verified: LT (specifically the JSON parser) stumbles over the < in the category description.

[172MC]

FYI I tried again using OpenSky with same result - mostly A320s - while ADS-B Exchange provided correct aircraft types

[TwinFan]

Found the reason...EDIT: not necessarily a bug:

Here (LTFlightData.h:138) the function "empty" expects and operator code to be set:

        // static data not yet properly filled?? (need acType and some operator info)
        inline bool empty() const { return acTypeIcao.empty() || (opIcao.empty() && op.empty()); }

This is included in the checks if data is valid. The data for A6CC0C contains no acTypeIcao! In the received network data it is "typecode":"". The human readable Model information is filled, but not the ICAO code.

So...refusing the data is correct...however, if we see that behaviour too often we might need to look for a different solution, probably a lookup in the Doc8643 data. That's why I also add label 'Enhancement' to it.

[172MC]

I am not sure I understand why it seems to affect OpenSky and not ADS-B Exchange

[TwinFan]

Well, it’s different databases maintained by different software and different people. Lots of chances to produce differences. It’s free (under conditions) resources...no warranty, no guarantees, not complete.

So we need to live with yet another inaccuracy. Unfortunately, “172S” doesn’t appear as a model description in Doc8643 either (maybe that’s also why OpenSky didn’t find it?). My best idea for the moment is to add a separate list of manufacturer/model to match to ICAO type code (kind of a reverse Doc8643) we can add to once we find such a problem. And where we can assign manufacturer defaults. If I see a Cessna and render a C172 I’m 80% right ;)

[172MC]

I understand those potential differences, but I seem to recall seeing a King Air 200 shown correctly as a BE20 and a Bombardier Challenger shown correctly as a CL30 by ADS-B Exchange and shown as A320s by OpenSky; could OpenSky just ignore GA/business aircraft (at KAPA - essentially a GA/business jet aitport, OpenSky only had airliners).

In any case, matching Beechcraft to either BE58 or BE9L in a reverse Doc8643 would make it right also 80% of the time.

[172MC]

Using ADS-B Exchange, I finally caught a displayed aircraft without ICAO type

Log.txt

[TwinFan]

Aha, ADS-B Exchange as well, soso. With ADS-B Exchange the a/c master data comes within the tracking data (that's why there is only one stream). So unlike OpenSky I cannot check the master data independently. But one line out of your Log.txt already tells most of what we need:

LiveTraffic 1544121748.1 INFO  src\LTAircraft.cpp:871/LTAircraft: Added aircraft A4DD8F (B737 SWA), a/c model 'B737 SWA B737_SWA', flight model [MediumJets], bearing 328, distance 14.2km
LiveTraffic 1544121748.1 INFO  src\LTAircraft.cpp:871/LTAircraft: Added aircraft A63952 ( ), a/c model 'A320 BAW A320_BAW', flight model [MediumJets], bearing 97, distance 10.2km

Note the empty paranthesis in the second line: That's the aircraft from your screenshot. That's where the type code should be as in the first of the log lines. So also here: ADS-B Exchange didn't send an ICAO type code. (BTW...out of curiosity I checked OpenSky's master data...they don't even know A63952.)

[TwinFan]

Didn't yet implement the reverse-DOC8643 thing...that'll take a few days. But as a first response to the issue and to more easily identify it the next version will come with a specific log message:

LiveTraffic 1544133701.6 WARN  LTChannel.cpp:1296/ProcessFetchedData: OpenSky Masterdata Online: Data for 'A6CC0C' lacks ICAO a/c type code, will be rendered with standard a/c A320

and the display of a/c info changed to at least display what's known:

[172MC]

Using 0.82, on clean XP11.26 at KAPA, Cessna Citation C510 appears as a BA airliner (does not appear to be an A320 ???). Need to add C510 on line 118 of related.txt

[TwinFan]

C510 added, good catch...I believe it was the only Citation code missing.

[172MC]

I was sure that I had reported Pilatus (PC6 and PC12) and Cirrus (SR220 and SR22) aircraft showing as A320s but I cannot find where. They are not in the related.txt, but are in Doc8643 of the latest LT v0.82. Here is one I caught this morning:

[TwinFan]

Well...certainly...we can add all the world's planes to related.txt...but that makes most sense for similar looking models. The multiplayer library should actually already look into the doc8643 data to find a match. For the case pictures in the screenshot I can clearly see from the info wnd (good to have that!) that the plane is properly identified with ICAO type SR20. It might not match any existing CSL model, it might not match a line in related.txt, but it should figure out from Doc8643 that an SR40 is just a L1P plane as any standard Cessna of which you definitely have something in your CSL package. That's not my code but deep in the guts of the multiplayer library...and there is something about "Special check - do NOT match a plane that isn't loaded" which looks fishy to me and might be from an old .acf era in which all planes must be actual real planes with .acf files. Or in other words: Looks to my like the entire idea to use the doc8643 file currently doesn't work. Well...just a wild guess from reading reading 10y old source code. Could you do me a favor: Can you by default switch on "Debug: Log model matching"? That will output some 5 to 10 lines of debug code per new aircraft with info how the model matching was done. With that output I could verify why the library didn't find anything reasonable. (Though it's damn easy to test...remove all traces of airbus and sit at a European airport...it should find a Boeing during the doc8643 match for all the Airbusses...will see...could be my next step for debugging.)

Edit: THat test was soooo easy: And confirmed: No Boeings displayed for airbusses. There's something wrong with the doc8643 matching in the multiplayer library...darn...I have to dig into that old code.

[172MC]

Easy on my side too: here is a Cirrus S20 shown as an A320 and the log.txt with Debug: Log model matching checked (I had it on all along but forgot to attach the log.txt in previous post)

Log.txt

[TwinFan]

Basically fixed! At least reinstalled the functionality originally intended. See here: An incoming A380 (what a concident after 2 minutes of testing) "correctly" matched to a B747:

LT MATCH - ICAO=A388 AIRLINE=SIA LIVERY=Singapore Airlines GROUP=
LT MATCH -    Group 0 key A388 SIA Singapore Airlines SIA
LT MATCH -    Group 1 key A388 SIA
LT MATCH -    Skipping 2 Due nil Group
LT MATCH -    Group 2 key  SIA Singapore Airlines SIA
LT MATCH -    Skipping 3 Due nil Group
LT MATCH -    Group 3 key  SIA
LT MATCH -    Group 4 key A388 SIA Singapore Airlines
LT MATCH -    Group 5 key A388
LT MATCH -    Skipping 6 Due nil Group
LT MATCH -    Group 6 key  SIA Singapore Airlines
LT MATCH -    Skipping 7 Due nil Group
LT MATCH -    Group 7 key 
LT MATCH - No match.
LT MATCH/acf - Looking for a  heavy L4J aircraft
LT Match/acf - matching WTC and configuration
LT MATCH/acf - found: B744
LiveTraffic 1544567161.3 INFO  LTAircraft.cpp:871/LTAircraft: Added aircraft 76CD74 (A388 SIA), a/c model 'B744 BAW B744_BAW', flight model [HugeAirbus], bearing 106, distance 6.8km

Liveries don't match up, defaults to BAW. Maybe I can even fix that with a recursive match...

[TwinFan]

Yea, also got the airline under control, A330-200 of China Eastern just landed in Sidney as flight CES711:

LT MATCH - ICAO=A332 AIRLINE=CES LIVERY=China Eastern Airlines GROUP=A332 A333 A338 A339
LT MATCH -    Group 0 key A332 CES China Eastern Airlines CES
LT MATCH -    Group 1 key A332 CES
LT MATCH -    Group 2 key A332 A333 A338 A339 CES China Eastern Airlines CES
LT MATCH -    Group 3 key A332 A333 A338 A339 CES
LT MATCH -    Group 4 key A332 CES China Eastern Airlines
LT MATCH -    Group 5 key A332
LT MATCH -    Group 6 key A332 A333 A338 A339 CES China Eastern Airlines
LT MATCH -    Group 7 key A332 A333 A338 A339
LT MATCH - No match.
LT MATCH/acf - Looking for a  heavy L2J aircraft
LT Match/acf - matching airline, WTC and configuration
LT MATCH/acf - found: B763 CES
LiveTraffic 1544571591.6 INFO  LTAircraft.cpp:871/LTAircraft: Added aircraft 7803BF (A332 CES), a/c model 'B763 CES B763_CES', flight model [HugeAirbus], bearing 171, distance 8.3km

displayed using a B763 of China Eastern...not too bad ;)

Time for a new beta release, I guess...but tomorrow.

[172MC]

"Time for a new beta release, I guess...but tomorrow." Fantastic! I can wait untilltomorrow,

PS Look at the CSLs loaded in my log.txt (3 comments above): I have added a lot of models that I have fixed so they do not produce parsing warning.

[CloudSkimmer]

Wow, @TwinFan, Sie sind schnell! Looking forward to that.

[TwinFan]

Issue still open for the idea of a "reverse doc8643", which would be considered in case icao type code is missing from the tracking data, but manufacturer/model text is available. The latter is human readable, not standardised, and would need an intelligent interpretation or damn long list for a mapping to reasonable ICAO type code. I'm more and more reluctant to implement that. With the above changes by far most planes are rendered reasonable now. The effort to maintain a "reverse doc8643" document will likely not be matched by the few remaining planes rendered with a default model. As a last change for now I enhanced the warning to include the man/mdl field information like this:

LiveTraffic 1547239996.3 WARN  LTChannel.cpp:1434/ProcessFetchedData: OpenSky Masterdata Online: Data for 'A3D039' (man 'Airbus', mdl 'A321-211') lacks ICAO a/c type code, will be rendered with standard a/c F16

which would allow to assess the situation over time and probably devise other means of deducting a good ICAO type code guess, e.g. by taking the info into the existing doc8643...the single example above could be fixed by stating: Try finding the manufacturer, ignore everything after a dash [which usually distinguishes sub-types of aircrafts not needed for a reasonable model match] and try to find a match in the type code and model fields of doc8643 (while ignoring spaces, dashes and the like on both sides). That rule would have determined A321 as a type code in this single case.

Will revisit these messages from time to time to see if a rule like the above really helps in a resonable number of cases.

[172MC]

I fully agree.

[172MC]

I saw some regression today. After getting LT v0.86.20190112, I flew from KAPA to KBJC in a new plane I just got (Carenado C172 G1000),and after landing, I was reloading several aircraft to link them to different profiles on my yoke) when I noticed one A320 high over the runway threshold before the take-off roll and it stayed at that level the whole length of the runway:

the regression is that it was a Piper PA28 that should not have been shown as an A320. Next I saw 2 other A320s landing and they were actually C172s, so the wrong model again.

Log.txt

[172MC]

PS Thinking that it might have been my error not to have made sure the CSLs in LT were loaded, I went back and reloaded at the same airport, went into LT settings/CSL, made sure the LT box is checked (there is only LT in this particular XP installation), clicked on Load anyway., got message that the CSLs were successfully loaded and still only A320s for GA

Log.txt

[172MC]

PPS On a hunch, I once again reloaded, went into LT settings and changed the default from A320 to C172 and now it seems that the C172 is used every time. It appears that the default overrides the model matching.

Log.txt

[172MC]

PPPS I really am confused now; I tried one more on a different XP installation. Now the model matching seems to work

Log.txt

The only difference between the 2 XP installations is that in the one where model matching did not work, I had LT showing NO label for aircraft (my thought here was I no longer test. I am just using LT to increase realism). I will now test that.

I am at a total loss, I hope you can make sense of everything above.

[TwinFan]

I can...it's just the original issue ;)

The real difference is not the installation but the received tracking data. Watch the "ICAO type" line in the info window!

In the first 2 cases it reads "?", i.e. the networks did not provide an ICAO type. LiveTraffic had no means of deriving a model and reverted to the default:

Which was A320 in both cases:

LiveTraffic 1547407407.9 WARN  src\LTChannel.cpp:1513/ProcessFetchedData: OpenSky Masterdata Online: Data for 'A6EA3E' (man 'Piper', mdl 'PA-28-181') lacks ICAO a/c type code, will be rendered with standard a/c A320
LiveTraffic 1547407495.5 INFO  src\LTAircraft.cpp:900/LTAircraft: Added aircraft A6EA3E ( N5448F), operator '', a/c model 'A320 BAW A320_BAW', flight model [MediumJets], bearing 318, distance 3.0km

LiveTraffic 1547408980.3 WARN  src\LTChannel.cpp:1513/ProcessFetchedData: OpenSky Masterdata Online: Data for 'A6EA3E' (man 'Piper', mdl 'PA-28-181') lacks ICAO a/c type code, will be rendered with standard a/c A320
LiveTraffic 1547409058.1 INFO  src\LTAircraft.cpp:900/LTAircraft: Added aircraft A6EA3E ( N5448F), operator '', a/c model 'A320 BAW A320_BAW', flight model [MediumJets], bearing 324, distance 3.3km

In the second and third case the ICAO type code actually is "C172", i.e. the networks did say that this actually is a C172, which certainly means that it will be rendered with C172, no matter what is configured as default (as you have a C172 CSL model in your installations). So for the case provided in the screenshot in case 3 the change from A320 to C172 did not even make a change.

LiveTraffic 1547410270.3 INFO  src\LTAircraft.cpp:900/LTAircraft: Added aircraft A14F69 (C172 N1839E), operator '', a/c model 'C172  C172_r2', flight model [GA], bearing 322, distance 11.5km

LiveTraffic 1547411336.9 INFO  src\LTAircraft.cpp:900/LTAircraft: Added aircraft ADF0BB (C172 N998DW), operator '', a/c model 'C172  C172_r2', flight model [GA], bearing 354, distance 5.1km

The changed default in case 3 did make a change for eight other aircrafts, though, not pictured in screenshots but visible from the Log.txt. That includes the Piper featured in the screenshots in the first 2 cases:

LiveTraffic 1547410189.8 WARN  src\LTChannel.cpp:1513/ProcessFetchedData: OpenSky Masterdata Online: Data for 'A6EA3E' (man 'Piper', mdl 'PA-28-181') lacks ICAO a/c type code, will be rendered with standard a/c C172
LiveTraffic 1547410245.8 INFO  src\LTAircraft.cpp:900/LTAircraft: Added aircraft A6EA3E ( N5448F), operator '', a/c model 'C172  C172_r2', flight model [MediumJets], bearing 342, distance 1.5km

All are now rendered using the C172 model. (The fact that the chosen flight model here is [MediumJets] is because that this the default based on no available ICAO type code...has nothing to do with the chosen CSL model. See the difference above for N1839E and N998DW from case 3 and 4: In their log lines the chosen flight model is [GA], which better matches a C172.)

Please don't get confused from the human-readable manufacturer/model information in the A/C info window, which is non-standardized. Our idea earlier here in the issue was to use that human-readable text for a reverse lookup... But as the reverse lookup is not yet implemented the entire model-matching bases solely on the ICAO type code. And that was empty in all irritating cases.

[172MC]

Thank you for the explanation. If I recall correctly, I was using only OpenSky as the source when the missing ICAO occurred (and I am not sure how that happened since I did not intentionally uncheck ADSBEX and have been using both at the same time consistently for a while) - I believe this is not the first time that OpenSky had this issue where I often fly.

[TwinFan]

Both OpenSky and ADS-B Ex have the issue from time to time and on different aircrafts. We can't do anything about it. If both channels are on and both channels see a specific plane then at least one of the channels need to pass in an ICAO code for the correct model to appear.

[172MC]

Understood. I suppose it is more the reason to use both, thereby reducing the odds of not having the ICAO code,.

[TwinFan]

Issue should have been further reduced though not at all solved by #97.

_{Sent with GitHawk}

[TwinFan]

Forum user crbascott extracted a useful file from OpenSky's aircraft.csv database, which does the trick of matching available model text to a/c type like this (some random extracts of his file):

MODEL|TYPECODE
17|NPOR
23|BE23
AS350BA ECUREUIL|AS50
AS350BB|G550
AS350D|AS50
AS350D ASTAR|AS50
LEISTUNG-SPATZ III|GLID
LEOPARD MOTH DH.85|DH85
LET CMELAK Z-37A|Z37P
LET L410|L410
LET L-410 FG|L410
A310 324ET|A310
A310 325|A310
A310-304 (AIRBUS)|A310
A310-308 (AIRBUS)|A310
A310-324|A310

This file is just under 300kb in size and can easily be distributed together with LiveTraffic. So the reverse-doc8643 functionality can now become reality:

• [ ] Read the file on startup (warning, but no error, if missing or unreadable)

• [ ] At the point where the final decision on a/c type is done right before creating the plane, if a/c type is yet unknown, look up the model info in above table and take the type info from there, informing via DEBUG-level message