search

rogers1000 / cyclingchaos

Cycling Data Package
6 stars 0 forks source link

UK National Racing Ingestions #39

Open rogers1000 opened 3 months ago

rogers1000 commented 3 months ago

Ingest data from British Cycling to get UK National level competitions.

Not looked into process yet.

rogers1000 commented 3 months ago

Events page is nested tables... shouldn't be too difficult but wasn't the easy ingestion I was hoping for.

rogers1000 commented 3 months ago

Calendar ingestion is using a semi-botch job as each page can only show 100 races. Need to manually input number of pages for the ingestion is work.

I'll put in some code that will calculate number of pages or find number of pages within webpage (using file soup).

Ingesting 2023 data only currently for both calendar and results.

rogers1000 commented 3 months ago

Waiting for confirmation that transformation has everything needed before signing off the current ingestion code for historical races.

Not done anything to do with future races yet, but imagine it will be very similar to historical results in terms of ingestion.

rogers1000 commented 2 months ago

Ingest isn't working as hoped. Need to look into hidden nested tables to get race classification and number of races within an event.

This would also give the race name for each race within an event.