To avoid spamming the scrapping targets, there needs to exist a caching workaround that does not utilize any in-memory caching libraries. Dumping the data in a JSON file would do the trick.
The dump file should be updated frequently, based on a scheduler of a 1-day interval would be preferable.
eoussama
commented
3 years ago
To avoid spamming the scrapping targets, there needs to exist a caching workaround that does not utilize any in-memory caching libraries. Dumping the data in a JSON file would do the trick.
The dump file should be updated frequently, based on a scheduler of a 1-day interval would be preferable.
[ ] Frequent updates to the dump file