search

jimmyday12 / fitzroy_data

2 stars 1 forks source link

rescrape afltables historical data #3

Closed peteowen1 closed 3 months ago

peteowen1 commented 3 months ago

scraped all historical afltables data to fix the issue: https://github.com/jimmyday12/fitzRoy/issues/214

only afldata and player_ids changed by running the rescrape

peteowen1 commented 3 months ago

actually - wait on this - seems to be some issues with id = 0

peteowen1 commented 3 months ago

Ok so realised fetch_player_stats_afltables() doesn't work well pre-2021 since IDs are fetched from https://[afltables.com/afl/stats/2021_stats.txt](https://afltables.com/afl/stats/2021_stats.txt) which only goes as far back as 2021

Where do the IDs before 2021 even come from? I see there's a df pulled from here git_url <- "https://raw.githubusercontent.com/jimmyday12/fitzroy_data/main/data-raw/afl_tables_playerstats/player_ids.csv" but not really sure how that gets built

jimmyday12 commented 3 months ago

That was a once off provided by Paul at AFL tables. That data isn't available on the site, and the endpoint he temporarily gave us at the time no longer works

peteowen1 commented 3 months ago

Ahhhh okay - ignore this pull request then as it rewrites the id table (as it does the weekly update script) Fix will have to be done another way