search

rafaelgomesxyz / esgst

An extension that enhances SteamGifts / SteamTrades.
MIT License
146 stars 23 forks source link

Cache Issue #1402

Closed Eiion closed 3 years ago

Eiion commented 5 years ago

Description When loading in pages (also with endless scrolling) on e.g. Groups, it takes quite long until all game info (as set in ESGST settings) is loaded in from Steam.

Steps to Reproduce

  1. Go to e.g. "Group" page
  2. Let pages be loaded in
  3. See error

Expected Behavior From my understanding specific game data should remain in the cache, after it has been requested from steam once, to speed up displaying all information. Therefore, if giveaways aren't new and have been displayed before, data should be there somewhat instantly.

Screenshots Screenshot

Additional Context As can be seen in the screenshot, information still gets requested from Steam before it's getting displayed altough the giveaways are older than three weeks and have been displayed many times before. This process can take quite long and even fail, which then can only be solved by reloading the pages - where once again all information has to be requested from steam.

Why is this? This isn't a new issue but with "old" ESGST from e.g. a year ago this process was really quick and typically never failed. If game data is indeed collected from cache, why can it fail? Why does it have to wait for other Game's data of other giveaways to be collected from Steam?

at46 commented 5 years ago

Afaik cache saves data only for seven days. Also the spinning hourglasses in your screenshot are all green which means ESGST has the data but it's close to be outdated and so ESGST will update it. I'm not sure about the time frame for this. Rafael gave some info about the colors and how it works here.

Eiion commented 5 years ago

Thanks for the links. Though game data like Trading Cards or Achievements typically doesn't change. So there should be no need to get that info after 7 days from Steam again. If it had e.g. Cards one time, it should immediately get the tag when loading the page. And then in the background there could be a check going on to see if it still has Cards. (Ok, Cards aren't the best example, but I guess you know what I want to say.)

Eiion commented 5 years ago

Another example:

Screenshot

Falling blocks is new and so info is not there yet (altough Game data from another giveaway of the same game most likely should be there) but Geneforge Saga isn't new. The giveaway is just a day old and all it's info was displayed perfectly before as I had loaded all group pages with all data in just minutes before and several times the day before. So when reloading the tab and all pages, data should not have to be gotten again from steam (presumably) if all game info is cached; which took so long that even all other pages had been loaded in before. All info should always be displayed instantly from cache. Changes should be checked in the background and info corrected accordingly if necessary in the first place.

Also, I don't mind if I have a big sized cache with all the game info I regularly request for games I have loaded on SG before. Sure, I don't know to what size the file could blow up, but I can't imagine it to be too big, considering it's just the games Name, ID, and a few more infos. A user setting adjusting a max. cache file size (with maybe a minimum setting to work well so people don't screw it up) before oldest entries get deleted wouldn't be a bad thing to have, I guess. So instead of deleting data of a certain age, delete it once a certain cache size is reached.

rafaelgomesxyz commented 5 years ago

Please do a test with endless scrolling paused. Load a single page, let all of the categories be fetched and then reload the page. The categories should appear instantly.

Eiion commented 5 years ago

I just did that and it's faster but not instantly. It should be just as quick though once all pages are already loaded with endless scrolling and then activating a specific filter. Lets see if it's still faster in a few hours, or if data again needs to be loaded.

Eiion commented 5 years ago

Just booted up the PC and went to Steamgifts' group page:

Screenshot

The "Shield Impact" giveaway is over 4 weeks old - how's there no data for this game already in the cache (that's what the green hourglass indicates, right?) considering how often data gets updated and how often I load that page? It should be there. (By the way, this is page 1.)

rafaelgomesxyz commented 5 years ago

Perhaps your cache is corrupt and it's not saving the correct date. I think I'll have to add some logs, because this is not reproduceable on my side.

Eiion commented 5 years ago

It hasn't always been like that though. Though it is an issue ever since timers where changed and the hourglass was introduced. It's something that started to annoy me so much due to the long waiting times that I decided to finally make it a topic, since it used to be quite instant for all the pages.

rafaelgomesxyz commented 5 years ago

Is this still happening? Can you check the size of your cache by running the code below on the console?

localStorage.esgst_gcCache.length

Eiion commented 5 years ago

1841537

rafaelgomesxyz commented 4 years ago

Is this still an issue?

kirara commented 4 years ago

While it seems to be somewhat faster now, some items are still reloading every time I open the home page (endless scrolling, about 400 items). Cache length = 2255814

Eiion commented 4 years ago

It was relatively good (i.e. better - still by far not as good as it used to be) over the past several weeks but started to be an issue again just sometime these past days. The steam web integration extension all is able to grab all the information basically in an instant. ESGST sometimes even fails to get it, so I tend to rely more on swi for some game information.

rafaelgomesxyz commented 3 years ago

Alright, I'm closing this because of #1701. If it's still a problem, let me know.

Eiion commented 3 years ago

It has been an issue for the past two weeks again. Before that it was running well for a few weeks or months.

rafaelgomesxyz commented 3 years ago

If you're talking about it being slow it's because the server was overloaded the past couple weeks, but it should be good again right now.

Eiion commented 3 years ago

Came here because I got an email notification about a new post from at46 saying "For me it seems that the cache isn't updated anymore since I upgraded to 8.8.7. The data gets fetched but when I open that page/giveaway the next time it is lost and ESGST will fetch it again. This is everywhere and I don't see any green hourglass anymore (all hourglasses are now red or yellow)." and wanted to mostly agree with it but it's not here anymore. Well, I guess my content fixed that.

at46 commented 3 years ago

After testing everything for a few times I was sure about the problem and posted here. But after that it looked like it would work again and I deleted the post. I should have edited it instead. I'm still unsure about the problem.