Closed orionw closed 2 months ago
@KennethEnevoldsen These are good questions! They both have the same answer in that the previous app.py file refreshed the leaderboard and created two intermediate objects (all_data_tasks
and boards_data
). I split it up so we can cache those intermediate objects, so I created a function to save them to file and load them without using pickle
since they were a very nested dictionary of dataframes.
I do think we should simply it at some point and only save out the files that are actually needed... and in a more understandable format. It would probably take some unravelling of the app.py
script to change it to use a more standard object format.
Thanks @orionw, exactly what I wanted to know!
To deal with Github LFS bandwidth, remove all pickle files and replace with regular data files. The results files are a bit complex but now they are also saved as readable formats.
@KennethEnevoldsen @Muennighoff thoughts?