Emersont1 / itchio

https://pypi.org/project/itchiodl/
MIT License
105 stars 17 forks source link

downloads empty folders without admin privileges / crashes with admin privileges or when trying to download individual games #69

Closed dph121 closed 1 year ago

dph121 commented 1 year ago

On Windows 11, with Python 3.9 or 3.11 and itchiodl 2.2.0, I've tried to download all the files to the "TTRPGs for Trans Rights in Florida" bundle that I bought.

If I enter "python -m itchiodl.downloader" in the command prompt without administrator privileges and enter my login information, folders get generated in my current directory with the names of items from the bundle as "downloading [bundle item]" is printed in quick succession. However, these folders remain empty.

If I enter "python -m itchiodl.bundle_tool" in the command prompt without administrator privileges and enter my login information and bundle link, it prints 'processing page [x]/[n]' several times, and I see no change. I'm not sure if this is expected.

If I enter "python -m itchiodl.downloader" in the command prompt with administrator privileges and enter my login information, it says

Error: 400 is not 200 Will print the response text (Please be careful as this may contain personal data or allow others to login to your account): {"errors":["Incorrect username or password"]} Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader__main.py", line 66, in main() File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader\main__.py", line 48, in main l = itchiodl.LoginAPI(user, password) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\login.py", line 43, in LoginAPI raise RuntimeError RuntimeError

If I enter "python -m itchiodl.bundle_tool" in the command prompt with administrator privileges and enter the relevant information, it says

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\bundle_tool__main__.py", line 17, in main() File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\bundle_tool__main__.py", line 13, in main b.load_games() File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\bundle.py", line 17, in load_games pages = int(s.select("span.pager_label a")[-1].text)


IndexError: list index out of range

If I try to download a single item, using "python -m itchiodl.downloader --download-game "https://housedok.itch.io/on-a-black-horse"" without admin privileges and enter my information, I get

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader__main.py", line 66, in main() File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader\main.py", line 58, in main lib.load_game(matches.group(1), matches.group(2)) File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\library.py", line 56, in load_game self.games.append(Game(k)) ^^^^^^^ File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\game.py", line 17, in init__ self.data = data["game"]


KeyError: 'game'

If I get admin privileges and try to download the same item, it gives the 'incorrect username or password' error instead.

I have the itch.io app and am logged in with it on this computer. I'm also logged in on itch.io on Chrome. I am able to download bundle items manually through both of these. I'm on a residential IP address. If you need any more information, I will be happy to supply it.

Emersont1 commented 1 year ago

Oh dear. Perhaps the API has changed. I'll look into this when I can

dvdcd commented 1 year ago

If I enter "python -m itchiodl.bundle_tool" in the command prompt without administrator privileges and enter my login information and bundle link, it prints 'processing page [x]/[n]' several times, and I see no change. I'm not sure if this is expected. I believe this is as intended, it should've added all the games to your itch library, and that is why the downloader is able to reference them

I'm experiencing a very similar issue, running itch-download or by running the main.py directly from the source, grabs all the files from my account and creates folders for them, but then immediately exits back to the command line without downloading any of the files. Looks like #70 is also a duplicate of this issue as well

I'm experiencing the same error @dph121 is while downloading a single game using `itch-download --download-game``, but I'm experiencing it in both Administrator and normal command prompt windows

Emersont1 commented 1 year ago

I can't seem to replicate it on my machine

I want to check that you're waiting for all the pages to load before it starts downloading (if your itch library is anything like mine, this may take some time)

(venv) [user@hostname ~]$ python -m itchiodl.downloader -k REDACTED
Loading page 1
Loading page 2
...
Downloading GAME NAME
Downloading GAME NAME

PS: Admin privileges are not required, so it should be run without them

Emersont1 commented 1 year ago

@dph121

If I enter "python -m itchiodl.downloader" in the command prompt with administrator privileges and enter my login information, it says

You're entering your username or password incorrectly, the error message says {"errors":["Incorrect username or password"]}. I'd strongly reccommend using an API key (from https://itch.io/user/settings/api-keys ) to make sure you don't get any authentication issues

If I try to download a single item, using "python -m itchiodl.downloader --download-game "https://housedok.itch.io/on-a-black-horse"" without admin privileges and enter my information, I get

Are you sure the game is in your library? if i try it with a game I own i get this

$python -m itchiodl.downloader --download-game https://bsl.itch.io/inkbit -k REDACTED
inkbit is a purchased game.
Loading page 1
Loading page 2
Downloading [Inkbit] Pixel Art Font Set
Downloading All TTF files (single download).zip
File Already Exists! All TTF files (single download).zip
Skipping [Inkbit] Pixel Art Font Set - All TTF files (single download).zip
Downloading Inkbit.ttf
File Already Exists! Inkbit.ttf
Skipping [Inkbit] Pixel Art Font Set - Inkbit.ttf
Downloading InkbitTwo.ttf
File Already Exists! InkbitTwo.ttf
Skipping [Inkbit] Pixel Art Font Set - InkbitTwo.ttf
Downloading InkbitThree.ttf
File Already Exists! InkbitThree.ttf
Skipping [Inkbit] Pixel Art Font Set - InkbitThree.ttf
Downloaded [Inkbit] Pixel Art Font Set (1 of 1)

What's your PWD, do you have write access to the folder?

dph121 commented 1 year ago

I'm putting what I think might be the most useful finding here at the top. I've attempted to answer the rest of your questions to the best of my ability below the horizontal line, though that is hopefully redundant.

I went around putting print statements in the code. If I add print(data) before line 17 in game.py and enter

python -m itchiodl.downloader -k REDACTED --download-game https://housedok.itch.io/on-a-black-horse

into the console, it gives the same error as before, but first prints

{'uploads': [{'storage': 'hosted', 'size': 3764109, 'game_id': 1955665, 'md5_hash': REDACTED, 'id': 7464733, 'position': 0, 'updated_at': '2023-03-05T15:59:49.000000000Z', 'type': 'default', 'filename': 'On A Black Horse - A Metahumans Rising One Shot.pdf', 'traits': {}, 'created_at': '2023-03-05T15:56:41.000000000Z'}]}

It looks like the code expects a different data format, with 'game' as the root instead of 'uploads' and some other changes as well. The same happens with other items I own:

{'uploads': [{'storage': 'hosted', 'size': 110406, 'type': 'default', 'id': 6474412, 'md5_hash': REDACTED, 'position': 0, 'updated_at': '2022-09-09T21:31:23.000000000Z', 'filename': 'By String and Song.pdf', 'game_id': 1697548, 'traits': {}, 'created_at': '2022-09-09T21:24:37.000000000Z'}]}

{'uploads': [{'created_at': '2022-09-29T14:45:32.000000000Z', 'storage': 'hosted', 'filename': "His Majesty's Monster Slayers v1.pdf", 'type': 'book', 'position': 0, 'game_id': 1724035, 'updated_at': '2022-09-29T14:48:44.000000000Z', 'id': 6574210, 'size': 588466, 'md5_hash': REDACTED, 'traits': {}}]}


Hopefully obsolete troubleshooting below:

I'd strongly reccommend using an API key (from https://itch.io/user/settings/api-keys ) to make sure you don't get any authentication issues

I've now tried with the API code. It gives the same error as before if I enter the API code correctly, and gives an API code error if it was entered incorrectly (so it does appear to be logging in to itch.io with the manual username and password)

I want to check that you're waiting for all the pages to load before it starts downloading (if your itch library is anything like mine, this may take some time)

I don't understand what you mean by this. When I run the program, there seems to be no pause or option to wait between 'loading page 12' and 'downloading šŸ€ On A Black Horse - A Metahumans Rising Adventure' in

C:\Users\flipw>python -m itchiodl.downloader -k REDACTED Loading page 1 Loading page 2 [...] Loading page 12 Downloading šŸ€ On A Black Horse - A Metahumans Rising Adventure Downloading Bookish Butterfly [etc.]

You're entering your username or password incorrectly, the error message says {"errors":["Incorrect username or password"]}.

I didn't (I checked, copy-pasting my password works iff I don't run it as administrator), but this only goes wrong if I'm running it as administrator, so it's perhaps not as important if that's not intended usage.

Are you sure the game is in your library?

I've currently manually downloaded "šŸ€ On A Black Horse" (https://housedok.itch.io/on-a-black-horse). I can see it in the itch.io default game folder (for me, C:\Users\flipw\AppData\Roaming\itch\apps\on-a-black-horse), and in the itch.io app under library / installed items. Nevertheless, I get:

python -m itchiodl.downloader -k REDACTED --download-game https://housedok.itch.io/on-a-black-horse Traceback (most recent call last): File "C:\Users\flipw\anaconda3\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\flipw\anaconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\downloader__main.py", line 66, in main() File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\downloader\main.py", line 58, in main lib.load_game(matches.group(1), matches.group(2)) File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\library.py", line 56, in load_game self.games.append(Game(k)) File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\game.py", line 17, in init__ self.data = data["game"] KeyError: 'game'

I can see "by string and song" (https://efangamez.itch.io/by-string-and-song-a-bard-only-ttrpg) in the itch.io app under library / owned items. I get the same error:

(base) C:\Windows\System32>python -m itchiodl.downloader -k REDACTED --download-game https://efangamez.itch.io/by-string-and-song-a-bard-only-ttrpg Traceback (most recent call last): File "C:\Users\flipw\anaconda3\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\flipw\anaconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\downloader__main.py", line 66, in main() File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\downloader\main.py", line 58, in main lib.load_game(matches.group(1), matches.group(2)) File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\library.py", line 56, in load_game self.games.append(Game(k)) File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\game.py", line 17, in init__ self.data = data["game"] KeyError: 'game'

What's your PWD,

If you mean password, this seems like a security hazard. If you mean something else, I don't know what it stands for.

do you have write access to the folder?

Yes. The folders get made by the base itchiodl.downloader, and I can manually make files in those folders as well as in the directory I'm operating in.

Emersont1 commented 1 year ago

Fixed by #72