search

m-rtijn / argostime

Keep an eye on prices
https://argostime.mrtijn.nl/
GNU Affero General Public License v3.0
8 stars 4 forks source link

Parallelize update script #61

Open m-rtijn opened 1 year ago

m-rtijn commented 1 year ago

If we start a separate thread per webshop, we can still keep the amount of requests/hour low per webshop, but still in a faster way check all products in the database

m-rtijn commented 1 year ago

The parallelization script is currently broken for MariaDB backends. This is due to mistakes in the SQLAlchemy backend handling.

Another possible improvement to the update script would be not to wait when no update was executed, or possibly just only querying not recently updated ProductOffers in the database.