ESGST marks reduced value games as full value games

[at46]

Description I synced reduced value games today and now it seems ESGST lost a lot of its data or synced incompletely. A lot of giveaways I won, made or see are wrongly labeled with full CV while being reduced CV.

Steps to Reproduce Sync Reduced and NoCV. Open any page with Giveaways.

Expected Behavior Show the correct info.

Console Errors Promise resolved after context unloaded esgst.js:2328

Screenshots

System (please complete the following information): ESGST: Extension v7.27.1 Browser: Firefox v62

Additional Context I tried to delete all data and sync again, restart the browser re-log in to sg but nothing helped. If I click the Full CV label on a game that is actually reduced CV it opens the bundles page and shows me that the game is in the list.

[at46]

The problem solved itself today.

[rafaelgomesxyz]

Sometimes an error may occur when updating the database. It's updated automatically every 24 hours, hence why it fixed itself.

Perhaps I should display some messages when that happens, and preserve the user's current database instead of wiping it.

[at46]

If I sync reduced cv games the list of games isn't directly fetched from the bundles page on sg?

[rafaelgomesxyz]

No, it's fetched from the database: https://docs.google.com/spreadsheets/d/1hdANpkPL_eKZHxJuvcmsVKbjtzN_2MXV2ZBMSJMkO-s/edit?usp=sharing

Imagine this: 5 users sync their reduced CV games. Let's say there are 200 bundle pages. There will be 1000 requests being made to SG in the next few minutes, which is very bad.

What actually happens: Only 200 requests are made to SG for the 200 bundle pages every 24 hours. The games are stored in the Google sheet. 5 users sync their reduced CV games. There will now be only 5 requests to Google instead of SG.

[at46]

Thanks for the explanation.

[cassolv]

@revilheart I understand that you can't just hammer SG with a ton of requests every other minute, of course, but is it feasible to either update that list slightly more often (say, every 3 or 4 hours) or maybe fetch the bundled list from SGTools for additions in the past 24 hours? When a game is first bundled, there is often a flood of GAs, and seeing all those as full CV can confuse the user a fair bit (and potentially impact the decision of entering or not entering the GA).

[rafaelgomesxyz]

@cassolv How often is the SGTools list updated?

[cassolv]

Good question. looks into it

According to the SG thread, "every 6 hours starting at 3:00AM GMT". That being the case, I imagine you could potentially do it every 6 hours starting at midnight GMT and then parse the SGTools list a bit over 3 hours later. You could try coordinating something with knsys (the person behind SGTools), too.

[rafaelgomesxyz]

The database will continue to be fully updated every 24 hours. I don't see the point in reducing that. But now every 6 hours the SGTools page will be fetched and any new games will be added. This change is effective immediately.

[cassolv]

That's a lot better than what we have now, so it works for me. Thanks. :+1: BTW, I assume you'll be fetching the SGTools page a bit after it updates (say 15 minutes later) to avoid race conditions?

Incidentally, are you adding any logic for detection of failure of the main update so that it suspends updates via SGTools until the next update (or uses the old version of the DB)? Otherwise, I think that might introduce a regression on this very issue: main update fails -> SGTools-based update adds some entries -> DB is not empty anymore (only almost) -> script thinks the list is valid and uses it -> this issue happens again (but for almost all games instead of all).

[rafaelgomesxyz]

I haven't checked the time yet, will do so later.

The main update no longer empties the database before updating, so there's no need for that detection.

[cassolv]

Oh, then that shouldn't cause problems. :100: