search

probberechts / soccerdata

⛏⚽ Scrape soccer data from Club Elo, ESPN, FBref, FiveThirtyEight, Football-Data.co.uk, FotMob, Sofascore, SoFIFA, Understat and WhoScored.
https://soccerdata.readthedocs.io/en/latest/
Other
511 stars 87 forks source link

[FBref] HTTP error 500 reading team match stats for current and past seasons, many leagues #604

Closed martinwilliamsiet closed 2 weeks ago

martinwilliamsiet commented 3 weeks ago

Hi!

I'm using Python 3.11.9 and soccerdata version 1.7.1.

The following code is returning "HTTPError: 500 Server Error: Internal Server Error"

ARGstats = sd.FBref(leagues="ARG-ArgentinePrimeraDivisión", seasons='2324')
ARGteamMatchMisc = ARGstats.read_team_match_stats(stat_type="misc")

err500sd

Same issue for other leagues, big 5 included, and with different stat types.

It appears that the URL that is being requested (https://fbref.com/en/squads/4bee7ba3/2023-2024/matchlogs/all_comps/misc) is incorrect for only team match stats, (https://fbref.com/en/squads/4bee7ba3/2024/matchlogs/all_comps/misc) also doesn't work for current season, but (https://fbref.com/en/squads/4bee7ba3/2023/matchlogs/all_comps/misc) and previous years, with just one year listed in link after squad identifier, all appear to open the correct page.

Thanks in advance!

Edit: I have set up start and end months for all leagues in league dictionary correctly.

Gabrynho commented 2 weeks ago

I'm facing the same problem scraping MLS data from the last 4 seasons, I can obtain the first 3 seasons correctly, but the current one raises the same error for every feature (stat_type) and I don't know how to fix it either.

probberechts commented 2 weeks ago

Note that the Argentinian league and MLS are not supported. If you want to scrape data for these leagues, it is up to you to adapt the code accordingly.

It seems to work fine for the big five leagues, both for the current and previous seasons.

import soccerdata as sd
fbref = sd.FBref("ENG-Premier League", [2223, 2324], no_cache=True)
fbref.read_team_match_stats(team="Manchester City")
probberechts commented 2 weeks ago

This might have been fixed as a side effect of #613.

martinwilliamsiet commented 1 week ago

Thanks! I attempted to fork and fix on my own but it caused 429 errors and was in the middle of figuring out why that happened. I am concatenating dataframes and pulling data from it and it may have been too much to handle with the source change. I'm unsure how to send you what I did, merging did not work for me. I will try again soon, thank you for the fix!