[BUG] Loading data from the 2021 season does not work anymore

[lalutir]

Describe the issue:

As previously reported by other people, the data from the 2022 season cannot be loaded anymore. However since last night, the same error started occurring for me for the 2021 season as well. The error is a KeyError on 'DriverNumber'

Reproduce the code example:

fp1 = ff1.get_session(2021, 1, 'FP1')
fp1.load()
fp1 = fp1.laps

Error message:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[15], line 2
      1 fp1 = ff1.get_session(2021, 1, 'FP1')
----> 2 fp1.load()
      3 fp1 = fp1.laps

File c:\Users\***\AppData\Local\Programs\Python\Python311\Lib\site-packages\fastf1\core.py:1424, in Session.load(self, laps, telemetry, weather, messages, livedata)
   1419 _logger.info(f"Loading data for "
   1420              f"{self.event['EventName']} - {self.name}"
   1421              f" [v{fastf1.__version__}]")
   1423 self._load_session_info(livedata=livedata)
-> 1424 self._load_drivers_results(livedata=livedata)
   1426 if self.f1_api_support:
   1427     if laps:

File c:\Users\***\AppData\Local\Programs\Python\Python311\Lib\site-packages\fastf1\core.py:2189, in Session._load_drivers_results(self, livedata)
   2185 # try loading from both sources if they are supported
   2186 # data is joined afterwards depending on availability
   2187 if self.f1_api_support:
   2188     # load driver info from f1 api
-> 2189     driver_info_f1 = self._drivers_from_f1_api(livedata=livedata)
   2190 if not self.event.is_testing():
   2191     # load driver info from ergast
   2192     driver_info_ergast = self._drivers_results_from_ergast(
   2193         load_drivers=True, load_results=True
   2194     )

File c:\Users\***\AppData\Local\Programs\Python\Python311\Lib\site-packages\fastf1\core.py:2273, in Session._drivers_from_f1_api(self, livedata)
   2270         for first, last in zip(driver_info['FirstName'],
   2271                                driver_info['LastName']):
   2272             driver_info['FullName'].append(f"{first} {last}")
-> 2273 return pd.DataFrame(driver_info, index=driver_info['DriverNumber'])

KeyError: 'DriverNumber'

[theOehrly]

Thanks for noticing. As far as I'm aware, F1 is still working on their cloud provider/server migration. Let's wait and see what happens there.

[ChemiF1]

Could be possible to create some kind of backup server? Just to get the data if the normal server fails

[Casper-Guo]

That will require access to the database that powers the F1 API backend. Even if we can reconstruct it, hosting that much data (all the telemetry for every session since 2018) can get expensive.

[theOehrly]

I'd really prefer not needing to host this. There's a bunch of reasons why. Cost and creating yet another thing that needs maintenance are just two examples.

[Casper-Guo]

From what I can see 2021 is back to working now. Loading 2022 data is still down.

[duanemck]

Is this issue affecting the current (2024) season as well? When querying, I only get meetings since the Spanish GP (21 June).

[theOehrly]

Is this issue affecting the current (2024) season as well? When querying, I only get meetings since the Spanish GP (21 June).

How/what are you querying exactly? With FastF1 you should only see this if you explicitly select the 'f1timing' backend when fetching the event schedule.

[duanemck]

I'm actually browsing the FastF1 codebase and then calling some of the API endpoints using Postman, just curious to see what's available. So yeah, I guess it would be the same as using the f1timing backend from within FastF1.

Apologies, did not mean to imply this was an issue with FastF1 with my original question, just thought the upstream F1 livetiming issues discussed here may be related to not all meetings coming back via their API.

[theOehrly]

@duanemck in that case, yes, the Index.json file that provides information about available sessions is broken for the 2024 season since the server migration. It is missing the earlier races of this season.

This is not relevant for the overall problem, though. FastF1 infers API paths from other information and generally assumes that the data exists. Any error that occurs right now is because there is actually relevant data missing.

One minor bug exists right now that prevents loading of a limited set of data when only some specific data is unavailable. This will be fixed soon.

There also will likely be an (experimental) solution to the problem in general within this week.

[theOehrly]

I have created a mirror server for the API now, as it seems like there is no other quick fix in sight. This is already implemented in the latest release, v3.4.0. At least one other large community project has taken the same approach, and I think it's good if overall the response to this problem is similar.

The server is hosted with Cloudflare Workers and their R2 object storage because their free tier quotas are high enough to run this on. But this was my first project with Cloudflare infrastructure, and I quickly set it up over the weekend. So please let me know if stuff isn't working. I might have messed something up.

The server currently only provides data for 2022 and 2021. The other seasons seem fine, so I haven't bothered with adding that data to the server as well. Please also let me know if there still are sessions that aren't working correctly.

The specific case where the /static/2024/Index.json endpoint is missing data is not fixed by this. This endpoint is usually not used by FastF1, and fixing this as well would be a bit more complicated. All other problematic endpoints (that I know of right now) return an error, and FastF1 simply falls back to the mirror if it receives an error response. I have decided that for now, this one endpoint isn't important enough to get special treatment.

In general, I'll consider this as an experimental feature with no guarantees whatsoever for now.