Heroic-Games-Launcher / HeroicGamesLauncher

A games launcher for GOG, Amazon and Epic Games for Linux, Windows and macOS.
https://heroicgameslauncher.com
GNU General Public License v3.0
7.94k stars 421 forks source link

Cannot load big GOG libraries (2400+ entries) #1485

Closed shakeyourbunny closed 1 year ago

shakeyourbunny commented 2 years ago

Describe the bug

Heroic Launcher seems to have really troube loading bigger game libraries, like mine, which has on GOG 2400+ entries.

Maybe it still works and is doing the initial scan, but there is no indication what it is doing.

Add logs

( don't know where the logs is )

Steps to reproduce

Expected behavior

Getting the games into the library.

Screenshots

No response

System Information

Additional information

Perhaps add a progress dialog, in what stages the scan is and how much it has progressed.

If you do a parse of the web site library, provide on what page the scan is and on what game; also on game x / num games, just like lgogdownloader.

This information could be provided in a non-modal window or something.

imLinguin commented 2 years ago

Progress bar is something to consider then. You can find logs under Settings > Logs. (Which will be helpful to find what really happened)

2400+ games - that's really big library 😯

shakeyourbunny commented 2 years ago

I tried the same procedure on my (beefier desktop computer) and did load after looking for 20 min on the spinning cursor (mind you my internet connection is 200 MBit downwards, so that should not a problem).

(12:58:22) INFO:    [Legendary]:       Running command : C:\Users\pulaski\AppData\Local\Programs\heroic\resources\app.asar.unpacked\build\bin\win32\legendary --version
(12:58:24) INFO:    [Legendary]:       Legendary location: C:\Users\pulaski\AppData\Local\Programs\heroic\resources\app.asar.unpacked\build\bin\win32\legendary
(12:58:24) INFO:    [Gog]:             GOGDL location: C:\Users\pulaski\AppData\Local\Programs\heroic\resources\app.asar.unpacked\build\bin\win32\gogdl
(12:58:24) INFO:    [Backend]:         

Heroic Version: 2.4.0-beta Caesar Clown
Legendary Version:  0.20.26 Dark Energy
OS: Microsoft Windows 10 Pro KERNEL: 10.0.19044 ARCH: x64
CPU: AMD Ryzen 5 3600 6-Core Processor @3.6 
RAM: Total: 31.93 GiB Available: 19.36 GiB
GRAPHICS: GPU0: AMD Radeon RX 6700 XT VRAM: 12272MB DRIVER:  

(12:58:25) INFO:    [Backend]:         Registered protocol with OS.
(12:58:25) INFO:    [Legendary]:       Installed game list updated
(12:58:26) INFO:    [Frontend]:        Refreshing Library
(12:58:26) INFO:    [Legendary]:       Refreshing library...
(12:58:26) INFO:    [Legendary]:       Refreshing Epic Games...
(12:58:27) INFO:    [Legendary]:       Running command : C:\Users\pulaski\AppData\Local\Programs\heroic\resources\app.asar.unpacked\build\bin\win32\legendary list
(12:58:28) INFO:    [Legendary]:       Updating game list
(12:58:28) INFO:    [Legendary]:       Game List Updated
(12:58:28) INFO:    [Gog]:             Found 0 game(s) to update
(12:59:52) INFO:    [Gog]:             Logging using GOG credentials
(12:59:52) INFO:    [Gog]:             Login Successful
(12:59:52) INFO:    [Gog]:             Getting data about the user
(12:59:52) INFO:    [Gog]:             Saved user data to config
(12:59:52) INFO:    [Frontend]:        Refreshing Library
(12:59:52) INFO:    [Gog]:             Getting GOG library
(12:59:53) INFO:    [Gog]:             Number of library pages: 24
(12:59:53) INFO:    [Gog]:             Getting data for page 2
(12:59:54) INFO:    [Gog]:             Getting data for page 3
(12:59:54) INFO:    [Gog]:             Getting data for page 4
(12:59:55) INFO:    [Gog]:             Getting data for page 5
(12:59:56) INFO:    [Gog]:             Getting data for page 6
(12:59:56) INFO:    [Gog]:             Getting data for page 7
(12:59:57) INFO:    [Gog]:             Getting data for page 8
(12:59:57) INFO:    [Gog]:             Getting data for page 9
(12:59:58) INFO:    [Gog]:             Getting data for page 10
(12:59:59) INFO:    [Gog]:             Getting data for page 11
(12:59:59) INFO:    [Gog]:             Getting data for page 12
(13:00:00) INFO:    [Gog]:             Getting data for page 13
(13:00:00) INFO:    [Gog]:             Getting data for page 14
(13:00:01) INFO:    [Gog]:             Getting data for page 15
(13:00:02) INFO:    [Gog]:             Getting data for page 16
(13:00:02) INFO:    [Gog]:             Getting data for page 17
(13:00:03) INFO:    [Gog]:             Getting data for page 18
(13:00:04) INFO:    [Gog]:             Getting data for page 19
(13:00:04) INFO:    [Gog]:             Getting data for page 20
(13:00:05) INFO:    [Gog]:             Getting data for page 21
(13:00:06) INFO:    [Gog]:             Getting data for page 22
(13:00:06) INFO:    [Gog]:             Getting data for page 23
(13:00:07) INFO:    [Gog]:             Getting data for page 24
(13:00:12) WARNING: [Gog]:             Unable to get covers from gamesdb for A Vampyre Story. Trying to get it from api.gog.com
(13:00:12) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:00:16) WARNING: [Gog]:             Unable to get covers from gamesdb for Agony UNRATED. Trying to get it from api.gog.com
(13:00:16) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:01:04) WARNING: [Gog]:             Unable to get covers from gamesdb for Cave Story®'s Secret Santa. Trying to get it from api.gog.com
(13:01:05) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:01:20) WARNING: [Gog]:             Unable to get covers from gamesdb for Cris Tales Demo. Trying to get it from api.gog.com
(13:01:20) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:01:25) WARNING: [Gog]:             Unable to get covers from gamesdb for Cyberpunk 2077 Goodies Collection. Trying to get it from api.gog.com
(13:01:25) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:01:54) WARNING: [Gog]:             Unable to get covers from gamesdb for Dr Livingstone, I Presume? Demo. Trying to get it from api.gog.com
(13:01:54) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:04:43) WARNING: [Gog]:             Unable to get covers from gamesdb for Leo the Lion's Puzzles. Trying to get it from api.gog.com
(13:05:41) WARNING: [Gog]:             Unable to get covers from gamesdb for Monster Bash HD. Trying to get it from api.gog.com
(13:06:03) WARNING: [Gog]:             Unable to get covers from gamesdb for Nightmares from the Deep 2: The Siren's Call. Trying to get it from api.gog.com
(13:06:03) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:06:04) WARNING: [Gog]:             Unable to get covers from gamesdb for Nightmares from the Deep 3: Davy Jones. Trying to get it from api.gog.com
(13:06:04) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:06:04) WARNING: [Gog]:             Unable to get covers from gamesdb for Nightmares from the Deep: The Cursed Heart. Trying to get it from api.gog.com
(13:06:04) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:07:49) INFO:    [Backend]:         Writing config for Heroic
(13:07:49) INFO:    [Backend]:         {
  "altLegendaryBin": "",
  "altGogdlBin": "",
  "addDesktopShortcuts": false,
  "addStartMenuShortcuts": false,
  "autoInstallDxvk": false,
  "autoInstallVkd3d": false,
  "customWinePaths": [],
  "defaultInstallPath": "C:\\Users\\pulaski\\Games\\Heroic",
  "defaultWinePrefix": "C:\\Users\\pulaski\\Games\\Heroic\\Prefixes",
  "disableController": true,
  "egsLinkedPath": "",
  "maxRecentGames": 5,
  "maxWorkers": 0,
  "minimizeOnLaunch": false,
  "nvidiaPrime": false,
  "otherOptions": "",
  "showFps": false,
  "showUnrealMarket": false,
  "useGameMode": false,
  "wineCrossoverBottle": "Heroic",
  "winePrefix": {},
  "wineVersion": {}
}
(13:08:27) WARNING: [Gog]:             Unable to get covers from gamesdb for Saint Kotar: The Yellow Mask. Trying to get it from api.gog.com
(13:08:27) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:09:50) WARNING: [Gog]:             Unable to get covers from gamesdb for Space Empires I. Trying to get it from api.gog.com
(13:09:50) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:09:51) WARNING: [Gog]:             Unable to get covers from gamesdb for Space Empires II. Trying to get it from api.gog.com
(13:09:51) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:09:51) WARNING: [Gog]:             Unable to get covers from gamesdb for Space Empires III. Trying to get it from api.gog.com
(13:09:51) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:09:53) WARNING: [Gog]:             Unable to get covers from gamesdb for Space Empires V. Trying to get it from api.gog.com
(13:09:53) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:10:09) WARNING: [Gog]:             Unable to get covers from gamesdb for Spring Sale Goodies Collection #1. Trying to get it from api.gog.com
(13:10:09) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:10:10) WARNING: [Gog]:             Unable to get covers from gamesdb for Spring Sale Goodies Collection #2. Trying to get it from api.gog.com
(13:10:10) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:10:48) WARNING: [Gog]:             Unable to get covers from gamesdb for Stranded Sails - Explorers of the Cursed Islands. Trying to get it from api.gog.com
(13:10:48) WARNING: [Gog]:             Unable to get covers from gamesdb for Stranger Things 3: The Game. Trying to get it from api.gog.com
(13:10:49) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:11:04) WARNING: [Gog]:             Unable to get covers from gamesdb for Summer & Winter: Olympic Challenge. Trying to get it from api.gog.com
(13:13:23) WARNING: [Gog]:             Unable to get covers from gamesdb for The Witcher Goodies Collection. Trying to get it from api.gog.com
(13:13:24) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:15:07) WARNING: [Gog]:             Unable to get covers from gamesdb for Wanderlust: Transsiberian. Trying to get it from api.gog.com
(13:15:17) WARNING: [Gog]:             Unable to get covers from gamesdb for Warhammer Skulls Digital Goodies. Trying to get it from api.gog.com
(13:15:17) WARNING: [Gog]:             Couldn't get info from api.gog.com, Using fallback vertical image
(13:16:39) INFO:    [Gog]:             Saved games data 

It would really be nice that something like a progress bar (alongside percentage; game / num games) on the library page appear during loading (at the bottom or so).

shakeyourbunny commented 2 years ago

look at the time stamps ;)

shakeyourbunny commented 2 years ago

Another option is - after you got the games list (2 min in) - that you already populate the library, but with default images and proceed to download the meta information. If the user arbitrarily clicks on a game during library load, override loading and get the clicked game info immediately.

Also, if the user closes the client prematurely before finishing downloading the metadata, just rescan the library for missing information an proceed downloading.

flavioislima commented 2 years ago

oof, 2400 games? are you the owner of GOG? although I think even them might not have such a big library 😅

like you said, one idea is to not download the covers and information until we load the list.

We have the same issue with the unreal marketplace right now, so we need a solution for both in this case.

LexRust commented 2 years ago

I managed to load my 3000+ GOG library on Steam Deck after 45min. Navigating library is a slog, but it works if you have the patience.

shakeyourbunny commented 2 years ago

On my desktop, loading time is really slow, @LexRust, but on my Steam Deck, the Heroic Launcher really kills the machine.

In order to improve loading time, I'd suggest loading the games in chunks. As far as I can tell, the application parses the library pages anyway, so would not be that much work just to load in the games during scanning the games .. and providing some sort of progress bar instead of staring at the debug log.

flavioislima commented 1 year ago

I think we need to develop a Lazy Load feature for the cards so we load a fixed number of them, like 25, 50 and then we load the rest but scrolling down or doing a search. Loading all at once like today is not sutainable.

arielj commented 1 year ago

this was addressed in Heroic 2.7

@shakeyourbunny can you confirm if this improved your loading times?