search

santosderek / Imgur-To-Folder

Download Imgur albums and images to desired folder
Other
80 stars 9 forks source link

Certain files can't be saved and stop the program #17

Open Fanfics opened 1 year ago

Fanfics commented 1 year ago

I'm very much not a coder, but while it was rough getting the scraper set up I did eventually get it up and running. However, it seems there are certain pages in my imgur favorites that the program is unable to save and which stop the process when it stumbles into them. I can get around them a little by guessing what page they're on and starting the process in the next one, but it's become clear there are a fair few and they each take out their entire page from the archive.

The specific error seems to be with something called the 'conversational dump', but actually understanding what's going wrong lies far beyond my grasp of programming lol.

Anything you've run into before? I've attach some pics with my name painted out if that's any help.

example 1 example 2

twang2000 commented 1 year ago

Looks like unintended characters in file extensions causes crash, as seen in '5gnSVQB - 1.jpg?1' with the "?1" being apparently a file versioning issue from what I looked up. "The extra characters, such as "?1", that you are seeing in your file extensions may be a result of file versioning. Some websites or services use version numbers or other additional information to keep track of different versions of the same file."

Here's a extremely unpleasant workaround. Please don't actually use this in production: 0.) Navigate to the folder where you initiated the download with "python setup.py install" (python3 is outdated, use python) 1.) Temporarily uninstall with "pip uninstall ImgurToFolder" 2.) Open the file imgur_download.py, in the imgurtofolder folder. On line 258, underneath "log.info('\t%s, File Size: %.2f MB' % (filename, file_size))" (Ctrl+F to find this), stick this cancerous mass in (python formatting): if '?' in filename[filename.rfind('.'):]: log.info("Found ? in {}".format(filename)) filename = filename[:filename.rfind('?')] log.info('Emergency patch done for ? stripping, {}'.format(filename)) 3.) Reinstall with "python setup.py install" Config files should still be there, so no need to do the OAuth again.

I hope this helps. Here's also just the raw "imgur_downloader.py" for those who just want to replace it. You'll still need to pip uninstall and then reinstall to update the package. imgur_downloader.zip

If there are any issues, please let me know.