wynick27 / steam-missing-covers-downloader

Adds missing library covers for new steam UI
Other
73 stars 13 forks source link

Downloader throws SyntaxError despite running successfully in the past #16

Open Rabbidking opened 4 years ago

Rabbidking commented 4 years ago

The .exe was working fine until recently, where for some unknown reason, I get this error upon launching it (some stuff censored out just in case):

Steam path: "my Steam path" SteamID: "my Steam ID" Steam grid path: "my grid path" Total packages in library: "number of packages" Retriving package details Loading packageinfo.vdf Traceback (most recent call last): File "missing_cover_downloader.py", line 580, in File "missing_cover_downloader.py", line 551, in main File "missing_cover_downloader.py", line 112, in get_missing_cover_app_dict File "missing_cover_downloader.py", line 197, in get_package_details File "missing_cover_downloader.py", line 214, in load_packageinfo File "vdf.py", line 524, in packageinfo_loads File "vdf.py", line 375, in binary_loads_at SyntaxError: Unknown data type at index 9680: b'\x19' [5932] Failed to execute script missing_cover_downloader

Again, all I've done is download the main executable/JSON package from the Releases tab. I have neither modified any files nor rebuilt anything from source. It used to work with no problems, but suddenly this error has been popping up, and I'm unaware of any fixes. I also tried deleting my packageinfo.vdf file from my appcache and letting Steam restore it on a restart, but that didn't seem to do anything.

TomKranenburg commented 4 years ago

Having the same identical issue

Jonius7 commented 4 years ago

It's because Steam changed its package format. See details here: https://github.com/SteamDatabase/SteamAppInfo

So that would mean editing vdf.py. Unfortunately I have no idea how to understand deserialising stuff at the byte level.

Jonius7 commented 4 years ago

Well, turns out I was able to understand enough and added 2 lines of code, which fixed the syntax error and allowed me to import 235 images into my Steam library.

The fix is on my fork if the pull request hasn't been accepted yet.

Gkont commented 4 years ago

Well, turns out I was able to understand enough and added 2 lines of code, which fixed the syntax error and allowed me to import 235 images into my Steam library.

The fix is on my fork if the pull request hasn't been accepted yet.

Do you think you could also supply a windows executable as in this repository ? Wouldn't know where to start compiling that. Thanks

Osyx commented 4 years ago

Do you think you could also supply a windows executable as in this repository ? Wouldn't know where to start compiling that. Thanks

You can download the files from his branch and run it like the readme says :)

Jonius7 commented 4 years ago

You can download the files from his branch and run it like the readme says :)

Speaking of which, the readme could use a little update, steam is at v1.0.2 now, so that pip install special note shouldn't be needed.

Do you think you could also supply a windows executable as in this repository ? Wouldn't know where to start compiling that. Thanks

I have added an unofficial exe to the "pre-release" (used PyInstaller btw if you were wondering) https://github.com/Jonius7/steam-missing-covers-downloader/releases/tag/new-format-fix

PatrickJnr commented 4 years ago

You can download the files from his branch and run it like the readme says :)

Speaking of which, the readme could use a little update, steam is at v1.0.2 now, so that pip install special note shouldn't be needed.

Do you think you could also supply a windows executable as in this repository ? Wouldn't know where to start compiling that. Thanks

I have added an unofficial exe to the "pre-release" (used PyInstaller btw if you were wondering) https://github.com/Jonius7/steam-missing-covers-downloader/releases/tag/new-format-fix

it says it's missing python38.dll when trying to launch it.

Jonius7 commented 4 years ago

it says it's missing python38.dll when trying to launch it.

Fixed it, needed to add --onefile flag to PyInstaller to package all the extra files into the .exe

Dsp211 commented 1 year ago

Hey, newbie here, would love to find a fix for this but this is happening again when trying to run the .exe on Jonius7's March 2021 version of the program as well, would it be possible to have a fix? Here's my error log (the exe closes itself)

Steam path: c:/steam SteamID: [redacted] Steam grid path: c:/steam/userdata/[redacted]/config/grid Traceback (most recent call last): File "missing_cover_downloader.py", line 580, in File "missing_cover_downloader.py", line 551, in main File "missing_cover_downloader.py", line 109, in get_missing_cover_app_dict File "missing_cover_downloader.py", line 221, in get_owned_packages KeyError: 'Licenses' [17920] Failed to execute script missing_cover_downloader

I'd try to fix it but my python skills are next to none, thank you!

Jonius7 commented 1 year ago

I've had a look and unfortunately I'm not sure how to fix it @wynick27 any idea? it seems for line 221 of missing_cover_downloader.py:

return list(int(pkgid) for pkgid in local_config['UserLocalConfigStore']['Licenses'].keys())

Licenses doesn't exist in the dictionary anymore

Jonius7 commented 1 year ago

I've found that changing it to

return list(int(pkgid) for pkgid in local_config['UserLocalConfigStore']['Software']['Valve']['Steam']['apps'].keys())

works for a bit but then gets stuck on load_appinfo

File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\vdf.py", line 375, in binary_loads_at
    raise SyntaxError("Unknown data type at index %d: %s" % (idx-1, repr(t)))
SyntaxError: Unknown data type at index 20: b'\x85'
wynick27 commented 1 year ago

It seems valve changed license info store location. Now licenses are stored in licensecache rather than localconfig.vdf. It's also encrypted with random number stream generated from user id.

wynick27 commented 1 year ago

@Jonius7 I've added parsing function for new licensecache format. Now it should work correctly.

Dsp211 commented 1 year ago

Will test this new version right now, thank you both so much!

Jonius7 commented 1 year ago

Still seems to be running into the same error @wynick27 :

Steam path: [steam path]
SteamID: [steam id]
Steam grid path: [steam grid path]
Total packages in library: 1336
Retriving package details
Loading packageinfo.vdf
Total packages in local cache 1336
Retriving apps in packages
Total apps in library: 2468
Retriving app details
Loading appinfo.vdf
Traceback (most recent call last):
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\missing_cover_downloader.py", line 584, in <module>
    main()
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\missing_cover_downloader.py", line 555, in main
    missing_cover_app_dict =  steam_data_reader.get_missing_cover_app_dict(not local_mode)
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\missing_cover_downloader.py", line 125, in get_missing_cover_app_dict
    owned_apps = self.get_app_details(owned_appids)
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\missing_cover_downloader.py", line 191, in get_app_details
    self.appinfo = self.load_appinfo()
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\missing_cover_downloader.py", line 207, in load_appinfo
    appinfo = vdf.appinfo_loads(f.read())
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\vdf.py", line 492, in appinfo_loads
    app = binary_loads(data[offset:offset+size])
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\vdf.py", line 274, in binary_loads
    result, idx = binary_loads_at(s,0,mapper,merge_duplicate_keys,alt_format)
  File "C:\Users\Jason\Downloads\steam-missing-covers-downloader\vdf.py", line 375, in binary_loads_at
    raise SyntaxError("Unknown data type at index %d: %s" % (idx-1, repr(t)))
SyntaxError: Unknown data type at index 20: b'\x85'
Jonius7 commented 1 year ago

@wynick27 Bump for update, something in the appinfo.vdf isn't able to be read.

I think I found some info here: https://github.com/SteamDatabase/SteamAppInfo

Jonius7 commented 1 year ago

I have found a fix to accomodate the new format, quote my old post, since the circumstances are the same:

Well, turns out I was able to understand enough and added 2 lines of code, which fixed the syntax error and allowed me to import 282 images into my Steam library.

Also fixed the asyncio RuntimeError: Event loop is closed error that happens at the end of downloading.

407e820ec43b61d1f4329a4703a92010b6707ff2


New version can be downloaded here: https://github.com/Jonius7/steam-missing-covers-downloader/releases/tag/2023_02