Closed lmolteno closed 3 years ago
Currently I do this by deleting the subjects rows, but this is bad
Now there's a set_flag.sh
script that you can run e.g.
./set_flag refresh
, and the next check of the checker
script will refresh the database
./set_flag rescrape
will tell the script to rescrape entirely.
There are problems when wanting to update custom content (#30), where the modified JSON file isn't within the docker image. Fix this by using a docker-compose volume.
e.g. Have a "flags" table, which has two columns "name", "status". If a "refresh", "True" row is added, then the scraping script will go through and refresh the database on the next run through.
Also, to improve uptime, maybe only clean the database just before we enter the new data in.