Wrong flight operation counts reported

[tfalas]

We use the excellent traffic python package for tracking ADS-B data from aircraft.

We have been using it during the last month for counting airport operations (takeoffs and landings) at particular airports in the USA, where ADS-B is mandatory, after January 1st 2020 with data from Opensky, We have noticed that the operations counted from Opensky with the traffic package are significantly less than the officially reported FAA operation counts.

Can you suggest any reasons that could lead to these discrepancies?

[xoolive]

Thank you for your kind words @tfalas

I wonder which data you are talking about and which FAA counts you are mentioning. It would indeed be interesting to document these discrepancies. With the elements you mention, I can only speculate...

If you talk about the dataset mentioned on this page https://traffic-viz.github.io/scenarios/covid19.html then please be aware that we found a bug in the data processing which was filtering out a significant number of flights.

If you are doing queries by yourself, "the final rule dictates that effective January 1, 2020, aircraft operating in airspace defined in 91.225 are required to have an Automatic Dependent Surveillance – Broadcast (ADS-B) system that includes a certified position source capable of meeting requirements defined in 91.227."

This means that aircraft which will not be flying in a subset of airspace may not be ADS-B equipped. I don't know much about the US airspace and am not sure about which share of aircraft are not ADS-B compliant but UAT compliant instead. See dump978 for reference.

Also, The OpenSky Network coverage in the US is not as complete as in Europe. This could be a source of investigation. If you play with the COVID19 dataset, we included first and last seen 3D positions of aircraft. It happens that a particular flight is not mapped to an airport, but the first/last 3D position is still in the vicinity of this airport.

Last but not least, if you find the serial id of receivers in the vicinity of airports, you may request on OpenSky Impala individual messages seen by these receivers and check which icao24 did not send any position to the network.

I will edit the message if I can think of more elements.

[tfalas]

Thanks for your reply @xoolive.

Here are some more details on the issue. I do not use the covid19 dataset. Instead I have downloaded the following:

from traffic.data import opensky
t = opensky.history("2020-01-01 00:00", "2020-01-31 23:59", airport = "KFRG")

KFRG is one of the airports where ADS-B is absolutely required after January 1st, 2020. So, in theory, the number of flights I get with

len(t)

(it returns 4034) should match what FAA is telling us from https://aspm.faa.gov/opsnet/sys/opsnet-server-x.asp (which returns 16845) for the same period at the same airport.

I have tried the same for another airport where ADS-B is required, with the same problem (actually worse, it only gives around a tenth of the actual number).

I am wondering whether the coverage at those airports may miss the signal when the aircraft fly very low (loss of line of sight by the receivers). Can you provide any indications on the algorithm used to detect a landing or a take-off from individual 3D point tracking?

[xoolive]

I tried to access the FAA link for more details but the link replies an empty page on my side.

Then, trying to investigate a bit... Let's look at this airport yesterday:

fl = opensky.flightlist("2020-07-09", arrival_airport = "KFRG")
fl.query('"2020-07-09 09:20-04:00" <= lastseen <= "2020-07-09 19:30-04:00"')

This is the result by OpenSky (70 flights total)

callsign	day	destination	origin	firstseen	icao24	lastseen
N72319	2020-07-09 00:00:00+00:00	KFRG	KBED	2020-07-09 12:44:38+00:00	a9b14e	2020-07-09 13:40:29+00:00
N38213	2020-07-09 00:00:00+00:00	KFRG	KFRG	2020-07-09 12:33:20+00:00	a4654f	2020-07-09 13:45:17+00:00
N269JJ	2020-07-09 00:00:00+00:00	KFRG	KHWV	2020-07-09 13:31:45+00:00	a2a13d	2020-07-09 13:45:37+00:00
N479DC	2020-07-09 00:00:00+00:00	KFRG	KFRG	2020-07-09 11:38:51+00:00	a5e2d7	2020-07-09 13:47:42+00:00
N356BZ	2020-07-09 00:00:00+00:00	KFRG	24OH	2020-07-09 12:53:20+00:00	a3fb48	2020-07-09 13:53:09+00:00
...	...	...	...	...	...	...
N55NY	2020-07-09 00:00:00+00:00	KFRG	KTEB	2020-07-09 22:49:48+00:00	a6fe55	2020-07-09 23:04:56+00:00
N82DB	2020-07-09 00:00:00+00:00	KFRG	NaN	2020-07-09 21:10:00+00:00	ab2ecf	2020-07-09 23:09:20+00:00
N6059D	2020-07-09 00:00:00+00:00	KFRG	KFRG	2020-07-09 22:30:22+00:00	a7dd69	2020-07-09 23:10:02+00:00
N53AA	2020-07-09 00:00:00+00:00	KFRG	NaN	2020-07-09 22:52:36+00:00	a6ae15	2020-07-09 23:16:08+00:00
N709DF	2020-07-09 00:00:00+00:00	KFRG	KFOK	2020-07-09 23:01:00+00:00	a9764e	2020-07-09 23:23:15+00:00

and for a comparison, this is what FlightRadar24 reports:

So here, it would look like OpenSky sees more of these flights.

As a matter of fact, there seems to be one receiver very closed to KFRG https://opensky-network.org/receiver-profile?s=1549351549 but the quality of their coverage seems to vary from day to day.

Also, N713SR does not appear in the OpenSky list: we saw her on her way to Plymouth, but not on the way in. Definitely no ADS-B data with this callsign.

opensky.flightlist("2020-07-09 00:00-04:00", "2020-07-10 00:00-04:00", callsign="N713SR")

callsign	day	destination	origin	firstseen	icao24	lastseen
N713SR	2020-07-09 00:00:00+00:00	KPYM	KFRG	2020-07-09 14:59:08+00:00	a988d2	2020-07-09 16:11:43+00:00

Looking at the way into KFRG from fr24 data:

• there seems to be a poor signal quality around KPYM; => I can't say whether the data they have for this flight is really ADS-B ? (faulty transponder but multilateration or US radar data?)

• it comes from the North into KFRG and the receiver is to the South of the field => maybe the coverage of low approaches from the North is not well covered

Well I am not sure of anything, maybe a way to investigate further could be to download data from a particular receiver with opensky.history(..., serials=["1549351549"]) and recode the landing airport detection (look at the quickstart page for inspiration.) It is actually also possible that OpenSky route detection algorithm fails when coverage is not great.

Please keep me posted with your results!

[nfala]

Xavier,

Thank you for all your work in answering our questions. You're right, some extra trouble shooting indicated that we will have to recode the landing airport detection. Our plan right now is to create a bounding box around the airport of interest rather than using the airport code to filter flights.

The following images show a flight from late yesterday (~2000 UTC, July 12) of an airplane doing touch and go practice at KLAF. The coverage was sufficient for the airport to be coded correctly, so we will also try to use this airport in parallel to do some more trouble shooting.

FlightRadar24 is not always able to code the departure/arrival airports, probably depending on coverage, so OpenSky is probably encountering the same challenges. Expanding our search and including a wider search box will potentially be less restrictive and provide a number that is closer to reality.

Here's the timeframe we used to get the flight of interest.

t = opensky.history(
    "2020-07-12 15:00",
    "2020-07-12 23:59",
    airport="KLAF"
)
t[4]

As for the FAA website, here's the correct link: https://aspm.faa.gov/opsnet/sys/Airport.asp It will give you the number of operations at any towered airport (during the control tower's operating hours). The website will create a report based on the parameters you feed into it (time period and airport identifier, in our case).

We'll let you know how things progress. Thanks again for looking into this!

Nicoletta

[xoolive]

Thank you Nicoletta for the detailed answer,

For your investigations, I recommend you go with the opensky.history(serials=) argument, just in case. At least try for a day and check whether you see some flights with altitude information but not latitude/longitude information (a smell for ADSB non compliant aircraft, still happening with commercial airliners in Europe... 😞 )

A simple way to check the receivers id is as follow:

f = t[4]
f.query_opensky_sensors()

You can check here: https://opensky-network.org/receiver-profile?s=-1408237224

Then you can iterate on flights and check if you get one without positional information:

t = opensky.history(
    start, stop, serials=[-1408237224]
)
non_compliant: Traffic = (
    t
    .iterate_lazy()
    .filter(lambda f: f.query("latitude == latitude") is None)
    .eval(desc="")
)

# or the old fashioned way:

for f in t:
    if f.query("latitude == latitude") is None:
        boom()

Just in case, if the library helps you get some work done for a publication, please mention it by email or with a PR in order to add it here: https://traffic-viz.github.io/publications.html

[xoolive]

Hello @tfalas do you think it would be reasonable to close this issue or is there any lesson to learn from this convo that should be included to the library?

[tfalas]

Thanks, we can close the issue.

On Sun, 6 Dec 2020 at 15:30, Xavier Olive notifications@github.com wrote:

Hello @tfalas https://github.com/tfalas do you think it would be reasonable to close this issue or is there any lesson to learn from this convo that should be included to the library?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xoolive/traffic/issues/89#issuecomment-739503027, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGWM2T7NTYXN6P4NVKSTHT3STOBQXANCNFSM4ORUFTXQ .