search

rachmadaniHaryono / we-get

:icecream: Command-line tool for searching torrents.
MIT License
153 stars 26 forks source link

fix decode error #42

Closed Sparviero-Sughero closed 2 years ago

Sparviero-Sughero commented 2 years ago

Status

READY

Migrations

YES

Description

somethime decode fail: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe5 in position 15054: invalid continuation byte my fix is to try also decode('latin-1')

Steps to Test or Reproduce

without fix, try this we-get --search "A Million Ways To Die In The West" --target limetorrents

rachmadaniHaryono commented 2 years ago

i don't think this is the right way.

rather than guessing the charset maybe it is time to use requests library and replace the urllib on this module

see https://stackoverflow.com/questions/4981977/how-to-handle-response-encoding-from-urllib-request-urlopen-to-avoid-typeerr

do you still want to work on this? if not i will rewrite this myself

Sparviero-Sughero commented 2 years ago

i can try but i'm not a py programmer :o(

rachmadaniHaryono commented 2 years ago

this is part of the solution

https://stackoverflow.com/a/36910311/1766261

but you have to make custom header just like previous code

here is requests custom header doc

https://docs.python-requests.org/en/latest/user/quickstart/#custom-headers