Emersont1 / itchio

https://pypi.org/project/itchiodl/
MIT License
105 stars 17 forks source link

Verbose Folders Argument (And Others) #58

Closed N64Core closed 1 year ago

N64Core commented 2 years ago

I feel like I've got these changes in a relatively clean spot for review, to summarize:

N64Core commented 2 years ago

I just noticed I left some debug code in here from when I was testing -sas , probably going to add some extra commits to this

N64Core commented 2 years ago

I'm also going to try and look into why it chokes on downloading specific files with unicode characters in their name, for example: https://feralphoenix.itch.io/soulsandstripes has soulsandstripes-日本語.pdf

N64Core commented 2 years ago

Fixed some issues that prevented the load_game and load_games functions from working properly in this branch, load_game now checks for a download key by running down your library and checking for that game's existence (I needed to use this to work on debugging the unicode issues), if the specified title isn't present it checks for free downloads from the game's page

N64Core commented 2 years ago

I requested another look at this PR but I suppose I should also attempt to summarize everything that's been changed / added in this branch:

  1. Added a -vf argument for creating a verbose folder structure that uses publisher and title display names as opposed to url slugs
  2. Added the -no-verify and -no-verify-file arguments, -no-verify-file prevents the script from creating additional files to store md5 hashes, and downloads will be verified against the md5 supplied by the itch.io api, no-verify skips all file verification entirely and simply assumes that files that share the same name are the same file.
  3. Added the -skip-above-size / -sas arguments, these will skip downloads that are above a certain supplied size in megabytes, supports decimal values.
  4. Added the -skip-library-load argument for the -download-game mode, by default the -download-game argument will now check across your library to see if download keys are present on your account, enabling this flag in essence means only the free titles on a given game will be downloaded.
  5. Fixed a number of issues relating to download stability and file creation, some are discussed in issue #57 , and created a mechanism for printing error stacks to a text file for review. This is accomplished by wrapping each instance of game.download() in a try-except-finally.
  6. Minor improvements to things like filepath generation and name sanitization, e.g. earlier in the commit history there was a particular issue where with -vf enabled, titles ending in ... would be skipped over entirely.
  7. Created a workaround for a bug that was apparently introduced in 2.1 where download keys were not properly being inherited by game objects as they were created, this issue created a lot of failed downloads.
  8. Additional stability improvements in areas where dictionaries were being accessed directly but it was possible for the value to not exist in the dictionary, creating an unhandled exception in that task. The instances where this was most commonplace now use dict.get() instead and should have better handling for non-existent values.

Additionally, this branch has only been tested against python 3.10.4

Emersont1 commented 1 year ago

So this really turned into an all encompassing PR. If any changes you still want merged in, could they be recreated as individual PRs

N64Core commented 1 year ago

So this really turned into an all encompassing PR. If any changes you still want merged in, could they be recreated as individual PRs

Hi! Yeah I can take a look at reviving some of these ideas as individual PRs

I kinda forgot about this :X