Open rogers1000 opened 8 months ago
Ingestion is done as part of race results data.
Need to do transform. Believe it will be part of file_soup_part2 where it will be a dynamic count of file_soup_part2 = file_soup.find_all('tbody')[0]
where 0 increases depending on jerseys available.
TTTs are causing issues... I have currently skipped any races that are TTTs using the race_results_df_master_errors
csv which collects any race which is TTTs or has stage cancellations.
gc_time_raw is now done for stage races.
Imagine that adding KOM, Points and Youth won't be too much trouble for Stage Races.
Not looked at GC Level but can't imagine too difficult to add.
Need to do GC Level first to identify which races have what jerseys. Then I can take the list of races with each jersey for the transformation on a stage-by-stage level.
Need to make change to the transform to check the table is actually looking at the classification it should be looking at.
A couple races don't do the standard: GC, KOM, Points, Youth which messed up transformation currently.
Should be a "Check Table name" and then transform.
issue about bringing in wrong classification fixed.
Youth jersey data transformation working.
Need to built in transformation relating to points and KOM scores
Transform has been completed. Need to do data validation to confirm for races which don't track Youth standings.
Accidentally broke results table! (All fixed now).
Still need to do data validation on races without youth jersey standings
Youth Standings are broken
KOM Standings were actually Points Standings
Did transforms for both points and KOM scores. Need to look into youth standings properly still.
Need to add from first place for both KOM and Points jerseys
Found a bug relating to KOM jersey data taking both points and kom data for the stage (see TDU 2023 Stage 1)
Youth is now working...
KOM is broken
KOM Position should be KOM_position_stage and Points Position should be Points_position_stage
Made those changes ^.
KOM investigation into why it shows both KOM and Points together needs to still happen.
Would be good to track KOM, Points and Youth Jerseys on Stage Races. Both for each stage and overall.