Exceen / 4chan-downloader

Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation
MIT License
281 stars 36 forks source link

"Something went wrong" #30

Closed necrorika closed 2 years ago

necrorika commented 3 years ago

Regardless of thread, board, script location, drive letter, attribute, or host machine the script returns "Something went wrong" and times out after several tries. image Above picture showing different threads and boards and different attribute flags, all three failing almost immediately. image Interestingly however it works fine on an arch install. I've tried it on two machines running Windows 10, reinstalled python 3 as well Not entirely sure why it's dropping almost immediately on Windows. Assume this is a Windows problem, but nothing changed since I used it yesterday. will close if I find a solution to using it on windows. sorry there isn't much information to go off of.

onioneffect commented 3 years ago

This was thrown by this block of code:

except (urllib.error.URLError, http.client.BadStatusLine, http.client.IncompleteRead):
    if not args.less:
        log.warning('Something went wrong')

In hindsight, it was silly not to include a statement right there to debug what the thing that "went wrong", so I'll include it in my own fork and you maybe you can post your results here?

necrorika commented 3 years ago

This was thrown by this block of code:

except (urllib.error.URLError, http.client.BadStatusLine, http.client.IncompleteRead):
    if not args.less:
        log.warning('Something went wrong')

In hindsight, it was silly not to include a statement right there to debug what the thing that "went wrong", so I'll include it in my own fork and you maybe you can post your results here?

Here ya go: image

onioneffect commented 3 years ago

It's not ideal, but try using http://boards.4chan.org/ instead of https. Maybe that'll work.

necrorika commented 3 years ago

Nope, same thing happens unfortunately image

onioneffect commented 3 years ago

Following some StackOverflow answers, I think you should run

pip install --upgrade certifi

This will upgrade your SSL certificates. If they're expired/corrupted, it can cause issues with urllib and Requests.

necrorika commented 3 years ago

Still returns the same errors. I upgraded the certificates and still the same thing, so I went and updated pip as well, still nothing. Went and added the trusted hosts to the pip.ini file, still the same thing as well so, overall, same thing.

onioneffect commented 3 years ago

Can you post the output of your

pip --version

and

python --version
necrorika commented 3 years ago

image looks like the most up to date python version is 3.9.7 so I'll probably update that in the morning if I have time and report back if all went well

necrorika commented 3 years ago

Little late, oops Now returns this error after updating to Python 3.10: image

pip and python versions: image

onioneffect commented 3 years ago

There must be a way to run the program without verifying the SSL certificates.

Exceen commented 3 years ago

@cherioux Could you please checkout this branch and tell me if it works? You need to run the script with the added -i/--ignore-ssl-verification-flag

https://github.com/Exceen/4chan-downloader/tree/ssl-verification

dridner commented 3 years ago

File ".\inb4404.py", line 43 if args.ignore_ssl_verification: ^ IndentationError: unindent does not match any outer indentation level

dridner commented 3 years ago

So I fixed the indent level, which was easy.

Got this.

Traceback (most recent call last): File ".\inb4404.py", line 192, in main() File ".\inb4404.py", line 32, in main download_thread(thread, args) File ".\inb4404.py", line 92, in download_thread html_result = load(thread_link).decode('utf-8') File ".\inb4404.py", line 51, in load ctx = ssl.create_default_context() NameError: name 'ssl' is not defined

Instead of import ssl in main(), I just declared at the top, and I don't have an issue any more. Is there anything wrong with just import ssl always? I'm a dev, just not a python dev. Not sure on semantics or anything.

Exceen commented 3 years ago

I'm really sorry, must have somehow uploaded something wrong. I fixed these import-issues now. Although it still needs testing if this flag fixes the SSL certificate issues.

dridner commented 3 years ago

Oh, guess I forgot to include that with this flag, it does indeed work!

necrorika commented 2 years ago

I'm late, but this does work!