Closed lmolteno closed 3 years ago
It only takes a minute to do the full scrape, and with optimisation I did in #37, it's pretty insignificant once the server is running, the only down-time in the database is during the actual entry through psycopg
commits.
These are exaggerated by the addition of resources to my database. I think that this would be a good place to implement a Rust solution. Multithreading!