UnicodeDecodeError when importing Epic Games Launcher games

[pythoninthegrass]

Hi @jaydenmilne,

Thanks for making this! Excited to use it to avoid opening the Epic Games Launcher as little as possible :)

Running into two separate issues:

1. The vdf library didn't get installed via pip install steamsync. Had to install it manually with pip install vdf. Not familiar with setuptools, but my guess is that it should use the imports from steamsync.py based on the setup.py script. That's the only dependency that didn't make it; maybe all the others are a part of the standard library?
2. More pressing is that the single game I have installed from Epic, "Control," raises a UnicodeDecodeError on line 199 trying to load the localconfig.vdf. Maybe this is related to the upstream issue https://github.com/ValvePython/vdf/issues/20? That was supposed to be fixed in https://github.com/ValvePython/vdf/pull/21.

Would attempt to add a try/except or set the encoding a la file = open(filename, encoding="utf8"), but not sure how to control (heh) for the error here.

Installing GTA V in Epic to see if it's a trend or just isolated to Control -- either due to the unicode string or possibly that Control hasn't dropped in Steam yet?

Scanning EGS manifest store (C:\ProgramData\Epic\EpicGamesLauncher\Data\Manifests)...
Collected 1 games from the EGS manifest store
Num | Game Name                 | App ID                           | Install Path
================================================================================================================
  1 | Control                   | Calluna                          | C:\Program Files\Epic Games\Control\Control.exe
Which games do you want to install (blank = all, or comma seperated list of numbers from table)?
:
Traceback (most recent call last):
  File ".\steamsync.py", line 327, in <module>
    accounts = enumerate_steam_accounts(args.steam_path)
  File ".\steamsync.py", line 199, in enumerate_steam_accounts
    cfg = vdf.load(localconfig)
  File "C:\Python38\lib\site-packages\vdf\__init__.py", line 198, in load
    return parse(fp, **kwargs)
  File "C:\Python38\lib\site-packages\vdf\__init__.py", line 96, in parse
    for lineno, line in enumerate(fp, 1):
  File "C:\Python38\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 6298: character maps to <undefined>

[pythoninthegrass]

Found the upstream vdf __init.py__ error handling:

if wide:
    result = result.decode('utf-16')
elif bytes is not str:
    result = result.decode('utf-8', 'replace')
else:
    try:
        result.decode('ascii')
    except:
        result = result.decode('utf-8', 'replace')

return result

[pythoninthegrass]

Same result with GTA V. So red herring with Control not being native to Steam.

Undefined byte 0x8d in position 6298 in localconfig.vdf. Let me know if you have suggestions for debugging locally. Cursory glance at the config shows that there's some sensitive info, otherwise I'd send it to you.

Looking at the tracebacks, might need to open a PR with vdf's maintainer for __init__.py lines 96 and 198 and/or modifying cp1252.py line 23.

Scanning EGS manifest store (C:\ProgramData\Epic\EpicGamesLauncher\Data\Manifests)...
Collected 2 games from the EGS manifest store
Num | Game Name                 | App ID                           | Install Path
================================================================================================================
  1 | Grand Theft Auto V        | 9d2d0eb64d5c44529cece33fe2a46482 | C:\Program Files\Epic Games\GTAV\PlayGTAV.exe
  2 | Control                   | Calluna                          | C:\Program Files\Epic Games\Control\Control.exe
Traceback (most recent call last):
  File "C:\Python38\lib\runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python38\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Users\l-ste\.vscode\extensions\ms-python.python-2020.5.78807\pythonFiles\lib\python\debugpy\no_wheels\debugpy\__main__.py", line 45, in <module>
    cli.main()
  File "c:\Users\l-ste\.vscode\extensions\ms-python.python-2020.5.78807\pythonFiles\lib\python\debugpy\no_wheels\debugpy/..\debugpy\server\cli.py", line 430, in main
    run()
  File "c:\Users\l-ste\.vscode\extensions\ms-python.python-2020.5.78807\pythonFiles\lib\python\debugpy\no_wheels\debugpy/..\debugpy\server\cli.py", line 267, in run_file
    runpy.run_path(options.target, run_name=compat.force_str("__main__"))
  File "C:\Python38\lib\runpy.py", line 263, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Python38\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Python38\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Users\l-ste\Documents\GitHub\steamsync\src\steamsync.py", line 327, in <module>
    accounts = enumerate_steam_accounts(args.steam_path)
  File "c:\Users\l-ste\Documents\GitHub\steamsync\src\steamsync.py", line 199, in enumerate_steam_accounts
    cfg = vdf.load(localconfig)
  File "C:\Python38\lib\site-packages\vdf\__init__.py", line 198, in load
    return parse(fp, **kwargs)
  File "C:\Python38\lib\site-packages\vdf\__init__.py", line 96, in parse
    for lineno, line in enumerate(fp, 1):
  File "C:\Python38\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 6298: character maps to <undefined>

[jaydenmilne]

GTAV won't work since it needs to be launched through the EGS so that you are authenticated. I have a fix incoming hopefully in a couple days, essentially we need to use a special URI as the shortcut target (make a shortcut in EGS and you'll see the format)

I thought I had setup.py install the dependency, I'll look closer at that.

Thanks for the interest, I'll look into this when I get a chance!

[pythoninthegrass]

Thanks so much, @jaydenmilne! No rush. I'll check back in a few days :D

[jaydenmilne]

OK, I think that I've resolved the dependency and decoding issues (thanks for doing the legwork of going upstream, that helped a ton!). Please update from pip and try it again.

To respond to your comments:

So red herring with Control not being native to Steam.

Actually, the error you are seeing is coming from parsing localconfig.vdf, which is only done to turn your numeric steam ID into a readable username. You could link to WINWORD.EXE and Steam would happily run it in big picture mode.

I've opened #2 for tracking GTAV not working. Let me know if this fixes your problem so I can close this issue.

[jaydenmilne]

As of version 0.2.0, GTAV should work as well. Let me know if it doesn't

[pythoninthegrass]

Tried both blank and selecting the index. Both raised a TypeError.

Two steps forward:

Scanning EGS manifest store (C:\ProgramData\Epic\EpicGamesLauncher\Data\Manifests)...
Collected 2 games from the EGS manifest store
Num | Game Name                 | App ID                           | Install Path
================================================================================================================    
  1 | Grand Theft Auto V        | 9d2d0eb64d5c44529cece33fe2a46482 | C:\Program Files\Epic Games\GTAV\PlayGTAV.exe  
  2 | Control                   | Calluna                          | C:\Program Files\Epic Games\Control\Control.exe
!!! SteamID for `<redacted>` not found.
Traceback (most recent call last):
  File "c:\Users\l-ste\Documents\GitHub\steamsync\src\steamsync.py", line 380, in <module>
    steamid = prompt_for_steam_account(accounts)
  File "c:\Users\l-ste\Documents\GitHub\steamsync\src\steamsync.py", line 236, in prompt_for_steam_account
    return accounts[0][0]
TypeError: 'SteamAccount' object is not subscriptable

[jaydenmilne]

OK, fixed that dumb typo, should work now 🤞 as of v0.2.1

[pythoninthegrass]

Thanks for your patience @jaydenmilne. We're getting there.

Scanning EGS manifest store (C:\ProgramData\Epic\EpicGamesLauncher\Data\Manifests)...
Collected 2 games from the EGS manifest store
Num | Game Name                 | App ID                           | Install Path
================================================================================================================
  1 | Grand Theft Auto V        | 9d2d0eb64d5c44529cece33fe2a46482 | C:\Program Files\Epic Games\GTAV\PlayGTAV.exe
  2 | Control                   | Calluna                          | C:\Program Files\Epic Games\Control\Control.exe
Which games do you want to install (blank = all, or comma seperated list of numbers from table)?
:
Installing shortcuts for SteamID `<redacted>`
Traceback (most recent call last):
  File ".\steamsync.py", line 394, in <module>
    add_games_to_shortcut_file(args.steam_path, steamid, games, args.live_dangerously, args.use_paths)
  File ".\steamsync.py", line 312, in add_games_to_shortcut_file
    all_paths.add(v["Exe"])
KeyError: 'Exe'

Also, went ahead and forked your program to use steamsync as a CLI program with the console_scripts entry point and style choices. Relevant edits are setup.py and steamsync.py.

Not used to contributing with git, but can attempt a pull request and think you should be able to merge what you like while discarding the rest. If that's not the case, might setup a branch with just relevant refactoring.

[jaydenmilne]

I should be thanking you for helping my janky script overcome "works on my machine", haha.

Hmm, this is the first one that isn't some silly mistake on my part, it appears that you have a shortcut in steam that has no executable path, interesting. I guess the schema of shortcuts.vdf isn't as rigid as I thought. I can add a check to make sure the key exists before reading it, but it seems to be a bandaid.

[jaydenmilne]

Let me know if 0.2.2 fixes it

[pythoninthegrass]

You're most welcome, @jaydenmilne!

Think 0.2.2 did the trick! Tested both Control and GTA V successfully. The script also found my Twitch shortcuts:

Scanning EGS manifest store (C:\ProgramData\Epic\EpicGamesLauncher\Data\Manifests)...
Collected 2 games from the EGS manifest store
Num | Game Name                 | App ID                           | Install Path
================================================================================================================
  1 | Grand Theft Auto V        | 9d2d0eb64d5c44529cece33fe2a46482 | C:\Program Files\Epic Games\GTAV\PlayGTAV.exe
  2 | Control                   | Calluna                          | C:\Program Files\Epic Games\Control\Control.exe
Which games do you want to install (blank = all, or comma seperated list of numbers from table)?
:
Installing shortcuts for SteamID `56951083`
Warning: Entry in shortcuts.vdf has no `Exe` field! Is this a malformed entry?
{'appname': 'SUPERHOT', 'exe': '"twitch://fuel-launch/62f505d5-9210-4784-9094-17cdc868f6da"', 'StartDir': 'C:\\Users\\l-ste\\AppData\\Roaming\\Microsoft\\Windows\\Start Menu\\Programs\\Twitch Games', 'icon': 'C:\\Users\\l-ste\\AppData\\Roaming\\Microsoft\\Windows\\Start Menu\\Programs\\Twitch Games\\SUPERHOT.url', 'ShortcutPath': '', 'LaunchOptions': '', 'IsHidden': 0, 'AllowDesktopConfig': 1, 'AllowOverlay': 1, 'OpenVR': 0, 'Devkit': 0, 'DevkitGameID': '', 'LastPlayTime': 0, 'tags': {}}
Warning: Entry in shortcuts.vdf has no `Exe` field! Is this a malformed entry?
{'appname': 'Aegis Defenders', 'exe': '"twitch://fuel-launch/5df60b68-ec08-4a80-bee6-7913ae0aad00"', 'StartDir': 'C:\\Users\\l-ste\\AppData\\Roaming\\Microsoft\\Windows\\Start Menu\\Programs\\Twitch Games', 'icon': '', 'ShortcutPath': '', 'LaunchOptions': '', 'IsHidden': 0, 'AllowDesktopConfig': 1, 'AllowOverlay': 1, 'OpenVR': 0, 'Devkit': 0, 'DevkitGameID': '', 'LastPlayTime': 0, 'tags': {}}
Added 2 new games
Backing up `shortcuts.vdf` to `C:\Program Files (x86)\Steam\userdata\<redacted>\config\shortcuts.vdf-20200521-213137.bak`
Updated `shortcuts.vdf` successfully!

➡   Restart Steam!
Done.

Lo and behold, the ephemeral nature of leasing games means both games are null and void. Amazon appears to have wiped out all the games I claimed :(

Think you're good to close out issue #1!