search

noaione / yuu

Yuu - a simple AbemaTV and other we(e)bsite video downloader
BSD 3-Clause "New" or "Revised" License
57 stars 10 forks source link

UnicodeEncodeError when parsing #30

Open zrenca opened 3 years ago

zrenca commented 3 years ago

hello, i tried to download several abema links and i have been getting this error after successfully auth. Thank you for taking your time to develop this for us :)

log output yuu download https://abema.tv/video/episode/somethingsomething -U email -P password -p 'socks5://127.0.0.1:1080'-o file.ts [INFO] Starting yuu v1.2.3... 2020-09-17 04:26:38 yuu -- [INFO]: Starting yuu v1.2.3... 2020-09-17 04:26:38 urllib3.connectionpool -- [DEBUG]: Starting new HTTPS connection (1): pastebin.com:443 2020-09-17 04:26:38 urllib3.connectionpool -- [DEBUG]: https://pastebin.com:443 "GET /raw/somthing HTTP/1.1" 200 None 2020-09-17 04:26:38 yuu -- [DEBUG]: Using proxy: socks5://127.0.0.1:1080 [INFO] AbemaTV: Authenticating 2020-09-17 04:26:38 yuu -- [INFO]: Authenticating [INFO] AbemaTV: AbemaTV: Fetching temporary token 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [INFO]: AbemaTV: Fetching temporary token 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Generated Device UUID: something 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Secret Key: key 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Generated applicationKeySecret: secret 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Sending json data 2020-09-17 04:26:38 urllib3.connectionpool -- [DEBUG]: Starting new HTTPS connection (1): api.abema.io:443 2020-09-17 04:26:40 urllib3.connectionpool -- [DEBUG]: https://api.abema.io:443 "POST /v1/users HTTP/1.1" 200 318 2020-09-17 04:26:40 yuu.abematv.AbemaTV -- [DEBUG]: Data sent, getting token 2020-09-17 04:26:40 yuu.abematv.AbemaTV -- [DEBUG]: User token: token 2020-09-17 04:26:42 urllib3.connectionpool -- [DEBUG]: https://api.abema.io:443 "POST /v1/auth/user/email HTTP/1.1" 200 403 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Authentication Token: token [INFO] AbemaTV: Parsing url 2020-09-17 04:26:42 yuu -- [INFO]: Parsing url 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Requesting data to Abema API 2020-09-17 04:26:42 urllib3.connectionpool -- [DEBUG]: https://api.abema.io:443 "GET /v1/video/programs/something HTTP/1.1" 200 1663 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Data requested 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Parsing json API 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: M3U8 Link: https://ds-vod-abematv.akamaized.net/program/something/1080/playlist.m3u8 yuu download https://abema.tv/video/episode/somethingsomething -U email -P password -p 'socks5://127.0.0.1:1080'-o file.ts [INFO] Starting yuu v1.2.3... 2020-09-17 04:26:38 yuu -- [INFO]: Starting yuu v1.2.3... 2020-09-17 04:26:38 urllib3.connectionpool -- [DEBUG]: Starting new HTTPS connection (1): pastebin.com:443 2020-09-17 04:26:38 urllib3.connectionpool -- [DEBUG]: https://pastebin.com:443 "GET /raw/Bt3ZLjfu HTTP/1.1" 200 None 2020-09-17 04:26:38 yuu -- [DEBUG]: Using proxy: socks5://127.0.0.1:10808 [INFO] AbemaTV: Authenticating 2020-09-17 04:26:38 yuu -- [INFO]: Authenticating [INFO] AbemaTV: AbemaTV: Fetching temporary token 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [INFO]: AbemaTV: Fetching temporary token 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Generated Device UUID: something 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Secret Key: key 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Generated applicationKeySecret: secret 2020-09-17 04:26:38 yuu.abematv.AbemaTV -- [DEBUG]: Sending json data 2020-09-17 04:26:38 urllib3.connectionpool -- [DEBUG]: Starting new HTTPS connection (1): api.abema.io:443 2020-09-17 04:26:40 urllib3.connectionpool -- [DEBUG]: https://api.abema.io:443 "POST /v1/users HTTP/1.1" 200 318 2020-09-17 04:26:40 yuu.abematv.AbemaTV -- [DEBUG]: Data sent, getting token 2020-09-17 04:26:40 yuu.abematv.AbemaTV -- [DEBUG]: User token: token 2020-09-17 04:26:42 urllib3.connectionpool -- [DEBUG]: https://api.abema.io:443 "POST /v1/auth/user/email HTTP/1.1" 200 403 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Authentication Token: token [INFO] AbemaTV: Parsing url 2020-09-17 04:26:42 yuu -- [INFO]: Parsing url 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Requesting data to Abema API 2020-09-17 04:26:42 urllib3.connectionpool -- [DEBUG]: https://api.abema.io:443 "GET /v1/video/programs/something HTTP/1.1" 200 1663 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Data requested 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: Parsing json API 2020-09-17 04:26:42 yuu.abematv.AbemaTV -- [DEBUG]: M3U8 Link: https://ds-vod-abematv.akamaized.net/program/something/1080/playlist.m3u8 Traceback (most recent call last): File "c:\python27\lib\runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "c:\python27\lib\runpy.py", line 72, in _run_code exec code in run_globals File "C:\Python27\Scripts\yuu.exe\__main__.py", line 9, in File "c:\python27\lib\site-packages\click\core.py", line 829, in __call__ return self.main(*args, **kwargs) File "c:\python27\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "c:\python27\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\python27\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "c:\python27\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs) File "c:\python27\lib\site-packages\yuu\command.py", line 138, in main_downloader outputs, reason = yuuParser.parse(res, resR) File "c:\python27\lib\site-packages\yuu\ext\abematv.py", line 427, in parse self.yuu_logger.debug('Video title: {}'.format(title)) UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)

log.txt

htc001120 commented 3 years ago

I got the same error, but the last message not exactlly the same

self.yuu_logger.debug('Title: {}'.format(output_name)) UnicodeEncodeError: 'ascii' codec can't encode character u'\u3010' in position 0: ordinal not in range(128)

I think the error is related to the title of the video

htc001120 commented 3 years ago

Just confirmed it's about the title, the script doesn't compatible with some title or some symbols,

I'm not familiar with Python I tried to skip the title part by providing an output file name, but the script seems trying to get the title anyway. I hardcoded the title part in the script and it runs well

zrenca commented 3 years ago

r some symbols,

I'm not familiar with Python

Thank you for the input, i was thinking about the same but I am such a noob of coding too. can you provide the change to the script? (>hardcoded the title part in the script)

Thank you

zrenca commented 3 years ago

I have partially fixed it by adding the following command to the file of command.pyand abematv.py, found on stackoverflow:

import sys reload(sys) sys.setdefaultencoding('utf8')

but it gives me this error after download and merge, the .ts video can still be played but I am not sure what this error is about:

[INFO][DOWN] AbemaTV: Finished downloading 2020-10-12 20:16:31 yuu -- [INFO]: Finished downloading [INFO][DOWN] AbemaTV: Merging video 2020-10-12 20:16:31 yuu -- [INFO]: Merging video Merging: 100%|####################################################################| 535/535 [00:44<00:00, 11.92file/s] [INFO][DOWN] AbemaTV: Finished downloading: filename.ts 2020-10-12 20:17:16 yuu -- [INFO]: Finished downloading: filename.ts [INFO][DOWN] AbemaTV: Parsing m3u8 and fetching video key for files no 2 2020-10-12 20:17:16 yuu -- [INFO]: Parsing m3u8 and fetching video key for files no 2 Traceback (most recent call last): File "/home/user/.local/bin/yuu", line 10, in sys.exit(cli()) File "/home/user/.local/lib/python2.7/site-packages/click/core.py", line 829, in call return self.main(args, kwargs) File "/home/user/.local/lib/python2.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/user/.local/lib/python2.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/user/.local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/user/.local/lib/python2.7/site-packages/click/core.py", line 610, in invoke return callback(args, **kwargs) File "/home/user/.local/lib/python2.7/site-packages/yuu/command.py", line 188, in main_downloader files, iv, ticket, reason = yuuParser.parse_m3u8(m3u8_list[pos]) IndexError: list index out of range