When running the current script on the cluster we do not get intermediate results so we don't know where we are / how long it will take the script to finish and it's also hard to deploy the model on different operating systems (Linux/MacOS). To solve this issue I want to change to script so that we can get more intermediate outputs and let it run on google colabs.
[x] Change main.py to output a DataFrame after each scraped race
When running the current script on the cluster we do not get intermediate results so we don't know where we are / how long it will take the script to finish and it's also hard to deploy the model on different operating systems (Linux/MacOS). To solve this issue I want to change to script so that we can get more intermediate outputs and let it run on google colabs.