probberechts / soccerdata

⛏⚽ Scrape soccer data from Club Elo, ESPN, FBref, FiveThirtyEight, Football-Data.co.uk, FotMob, Sofascore, SoFIFA, Understat and WhoScored.
https://soccerdata.readthedocs.io/en/latest/
Other
529 stars 90 forks source link

[FBref] Optimizing read_team_match_stats when data has been previously cached #458

Closed Kalaweksh closed 3 months ago

Kalaweksh commented 6 months ago

The method currently reads all cached html match log files whenever loading data, which is time and memory-intensive. A less expensive alternative would be to store an extra cache of previously loaded data (for instance, by calling .to_csv or .to_parquet on the returned Dataframes), increasing the performance of subsequent calls to the method.