search

Wingysam / Christmas-Community

Christmas lists for families
GNU Affero General Public License v3.0
212 stars 39 forks source link

Request: Automatically Update Item Prices #62

Open OneRadRedDad opened 1 year ago

OneRadRedDad commented 1 year ago

Currently, an item's price is pulled from the source website when it is first added to a user's wishlist and never updated again. This can result in inaccurate prices in CC, especially for items added around the Holidays, which tends to see more volatile prices.

If the items on a list could be pinged whenever a list is loaded to have prices updated, that would be beneficial.

Wingysam commented 1 year ago

It couldn't be on every page load because then loading a wishlist would take several minutes in some cases. Christmas Community could automatically update data continuously rather than a permanent property in the database. The primary concern with this is bot detection. Some sites will probably ban your IP if you constantly check items in the same way.

I'm willing to give continuously updating product data a shot, I'll add it behind an environment variable at some point soon.

OneRadRedDad commented 1 year ago

Fair point. How much is added to load time to update one price? What if the page loaded first, and then a loading animation above the price column while the prices updated afterwards?

If you go the continuos uodating route, I think even updating prices once per day would be sufficient

On Sat, Dec 10, 2022, 12:00 PM Wingy @.***> wrote:

It couldn't be on every page load because then loading a wishlist would take several minutes in some cases. Christmas Community could automatically update data continuously rather than a permanent property in the database. The primary concern with this is bot detection. Some sites will probably ban your IP if you constantly check items in the same way.

I'm willing to give continuously updating product data a shot, I'll add it behind an environment variable at some point soon.

— Reply to this email directly, view it on GitHub https://github.com/Wingysam/Christmas-Community/issues/62#issuecomment-1345315470, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4MCTNYIWGERBEFNQKZUPXLWMSZJLANCNFSM6AAAAAAS2ANHPI . You are receiving this because you authored the thread.Message ID: @.***>

Wingysam commented 1 year ago

I don't think it's okay to flood requests to every retailer in the wishlist every page load. There needs to be at least some caching and staggered requests. There should be a last update field in the database and once that reaches 24 hours, it should be in the queue for updates to data. One item should be updated every minute if the queue is not empty.