search

vradarserver / vrs

A .NET web server that can plot the positions of aircraft on a map
http://www.virtualradarserver.co.uk/
Other
267 stars 50 forks source link

ICAO with lowercase letters enters sqb database file #95

Open marcus-aa opened 1 year ago

marcus-aa commented 1 year ago

In some cases this can happen, not sure how or why.

image

marcus-aa commented 1 year ago

As well as some with tildes, I have around 4000 of these.

image

image

It seems some kind of process does not fully check the validity of incoming data. It could possibly be from aircraftlist-json receivers?

vradarserver commented 1 year ago

The database writer only saves one feed. What is the format of the feed that is being recorded?

The database writer plugin would filter those tilde ICAOs out. That filtering code has been in there since at least November 2016 - probably earlier than that, I think Nov 2016 was just when I made it a bit quicker.

Was this SQB pre-populated? Were the tilde codes in the original version of the database? Likewise were the lower-case ICAO24s also in there?

Also, is there anything else writing to the SQB file?

The plugin doesn't convert the ICAO24 to upper-case before it writes it, but I think it should be getting converted to upper-case before it gets as far as the plugin. Not 100% sure, I'd need to go through the code.

If the feed format is raw then it will definitely be in upper-case before it makes it as far as the plugin.

If the feed format is cooked then there's a possibility that the plugin is seeing the messages from the feed in their original state, in which case if the ICAOs are in lower-case on the feed then they might also be written in lower-case. If that's happening then that wouldn't be ideal, the rest of VRS only deals in upper-case ICAOs and BaseStation.SQB has a schema that is mostly case-sensitive. If that is happening then it would be good to know the format of the feed that's being recorded please.

But I think the fact that you have those tilde codes points at something else writing to the file, I can't see how they could have made it through the database writer plugin.

marcus-aa commented 1 year ago

Hi,

Only VRS (and my manual db updates) writes to the SQB. It has been in use by various versions of VRS since 2014 or something like that. But those records were created in the last 2-3 years, and certainly not imported from any other system.

The writer uses the combined feed of all receivers, varying formats, raw, basestation & json, so it is certainly "cooked".

I would suggest that the lower-case (and maybe the tildes) are entering through a json-format receiver, we are using our own json feed clients into VRS to save bandwidth, and while I have written a converter to upper-case in that feed client, it may have a bug, or something slipping through. Anyway, it would be good to have some checking on the input side, because a faulty feed could make a lot of damage to the database. Tilde-starting ICAOs are also common in the dumo json output unless we opt out of them from FA for example.

Maybe this could point you in the right direction, happy to help with any debugging.

ua4wiy commented 1 year ago

Such HEX was issued by FlightFeeder from flightaware. In this way, they hid secret and military aircraft from being shown in the VRS in the on-line MLAT mode.

vradarserver commented 1 year ago

@marcus-aa Sorry for the delay. I'm currently (slowly) porting the BaseStation database code to .NET Core and I noticed in there that one of the functions that gets used to create aircraft records will let lower-case ICAOs through to the database if it gets presented with one.

I've fixed that issue in v2 (a6e099e), v3 (139a2e8) and the upcoming .NET Core (7b3cf17) versions but I've not deployed anything yet.