search

xenova / chat-downloader

A simple tool used to retrieve chat messages from livestreams, videos, clips and past broadcasts. No authentication needed!
https://chat-downloader.readthedocs.io/
MIT License
906 stars 127 forks source link

[WARNING] Retry #-- (sleep for --s or press Enter). None (UnexpectedHTML) #189

Closed hellishvictor closed 1 year ago

hellishvictor commented 1 year ago

Hi, today chat_replay_downloader it started to show the following:

[INFO] Site: twitch.tv [INFO] Retrieving chat for "gameplay". [WARNING] Retry #1 (sleep for 0s or press Enter). None (UnexpectedHTML) [WARNING] Retry #2 (sleep for 1s or press Enter). None (UnexpectedHTML) [WARNING] Retry #3 (sleep for 2s or press Enter). None (UnexpectedHTML) [WARNING] Retry #4 (sleep for 4s or press Enter). None (UnexpectedHTML)

It always as worked great, till now. Cheers.

xenova commented 1 year ago

Please supply the full command you ran (including the -v tag).

hellishvictor commented 1 year ago

chat_replay_downloader.exe -v https://www.twitch.tv/videos/1673754110 > chat.txt

[DEBUG] Python version: 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64 )] [DEBUG] Program version: 0.0.9 [DEBUG] Starting new HTTPS connection (1): badges.twitch.tv:443 [DEBUG] https://badges.twitch.tv:443 "GET /v1/badges/global/display HTTP/1.1" 200 99976 [INFO] Site: twitch.tv [DEBUG] Parameters: {'url': 'https://www.twitch.tv/videos/1673754110', 'start_time': None, 'end_time ': None, 'max_attempts': 15, 'retry_timeout': None, 'timeout': None, 'max_messages': None, 'logging' : 'debug', 'pause_on_debug': False, 'inactivity_timeout': None, 'message_groups': ['messages'], 'mes sage_types': None, 'format': 'twitch', 'format_file': None, 'chat_type': 'live', 'message_receive_ti meout': 0.1, 'buffer_size': 4096} [DEBUG] Starting new HTTPS connection (1): gql.twitch.tv:443 [DEBUG] https://gql.twitch.tv:443 "POST /gql HTTP/1.1" 200 839 [DEBUG] https://badges.twitch.tv:443 "GET /v1/badges/channels/44445592/display HTTP/1.1" 200 25667 [DEBUG] Chat information: {'chat': <generator object TwitchChatDownloader._get_chat_messages_byvod id at 0x000000000372B5F0>, 'title': "curly hair comeback :') overwatch today with aria !", 'duration ': 19422, 'is_live': False, 'start_time': None, 'site': <chat_replay_downloader.sites.twitch.TwitchC hatDownloader object at 0x0000000003939C10>, 'format': <function ChatDownloader.get_chat..<l ambda> at 0x0000000003686820>} [INFO] Retrieving chat for "curly hair comeback :') overwatch today with aria !". [DEBUG] Starting new HTTPS connection (1): api.twitch.tv:443 [DEBUG] https://api.twitch.tv:443 "GET /v5/videos/1673754110/comments?client_id=kimne78kx3ncx6brgo4m v6wki5h1ko&cursor=&content_offset_seconds=0 HTTP/1.1" 410 0 [WARNING] Retry #1 (sleep for 0s or press Enter). None (UnexpectedHTML) [DEBUG] [DEBUG] https://api.twitch.tv:443 "GET /v5/videos/1673754110/comments?client_id=kimne78kx3ncx6brgo4m v6wki5h1ko&cursor=&content_offset_seconds=0 HTTP/1.1" 410 0 [WARNING] Retry #2 (sleep for 1s or press Enter). None (UnexpectedHTML) [DEBUG] [DEBUG] https://api.twitch.tv:443 "GET /v5/videos/1673754110/comments?client_id=kimne78kx3ncx6brgo4m v6wki5h1ko&cursor=&content_offset_seconds=0 HTTP/1.1" 410 0 [WARNING] Retry #3 (sleep for 2s or press Enter). None (UnexpectedHTML) [DEBUG] [DEBUG] https://api.twitch.tv:443 "GET /v5/videos/1673754110/comments?client_id=kimne78kx3ncx6brgo4m v6wki5h1ko&cursor=&content_offset_seconds=0 HTTP/1.1" 410 0 [WARNING] Retry #4 (sleep for 4s or press Enter). None (UnexpectedHTML) [DEBUG] [DEBUG] https://api.twitch.tv:443 "GET /v5/videos/1673754110/comments?client_id=kimne78kx3ncx6brgo4m v6wki5h1ko&cursor=&content_offset_seconds=0 HTTP/1.1" 410 0 [WARNING] Retry #5 (sleep for 8s or press Enter). None (UnexpectedHTML) [DEBUG]

xenova commented 1 year ago

Right... you are using a very outdated version of the software (v0.0.9 vs 0.2.3), which still uses the v5 endpoint (discontinued as of a few days ago: see https://github.com/xenova/chat-downloader/issues/186)

First, uninstall chat-replay-downloader using 

pip uninstall chat-replay-downloader

then install chat-downloader using

pip install chat-downloader

You can then use the same commands as before (replacing chat_replay_downloader with chat_downloader. For example,

chat_downloader https://www.twitch.tv/videos/1673754110 > chat.txt

Also, to write text to a file, I would recommend using the -o (output flag for writing to a file instead of >.

For example, the command would become:

chat_downloader https://www.twitch.tv/videos/1673754110 -o chat.txt

This is because windows has errors with outputting unicode characters with > (either omitting, or crashing sometimes).

hellishvictor commented 1 year ago

Right... you are using a very outdated version of the software (v0.0.9 vs 0.2.3), which still uses the v5 endpoint (discontinued as of a few days ago: see #186)

First, uninstall chat-replay-downloader using

pip uninstall chat-replay-downloader

then install chat-downloader using

pip install chat-downloader

You can then use the same commands as before (replacing chat_replay_downloader with chat_downloader. For example,

chat_downloader https://www.twitch.tv/videos/1673754110 > chat.txt

Thank you very much for the step by step explanation, now it works like a charm :)

Also, to write text to a file, I would recommend using the -o (output flag for writing to a file instead of >.

For example, the command would become:

chat_downloader https://www.twitch.tv/videos/1673754110 -o chat.txt

This is because windows has errors with outputting unicode characters with > (either omitting, or crashing sometimes).

Using the -o flag it output the chat on very wide CSV xml format, but with the > there's not need to use a pre-configured file or anything since its just the time, name of the user and the text, and that for me is perfect,

Cheers.

xenova commented 1 year ago

I changed the default output format (-o chat.txt) so that it matches what is output by > chat.txt. Just test it out and see if it works :)

Otherwise, glad it's working again 👍

hellishvictor commented 1 year ago

I changed the default output format (-o chat.txt) so that it matches what is output by > chat.txt.

I've test it out, and using the -o has errors with outputting unicode characters, like ñ and accented á é í ó ú, but not problems at all with >, it also hides the full chat output on the console, which is something I prefer.

xenova commented 1 year ago

I've test it out, and using the -o has errors with outputting unicode characters, like ñ and accented á é í ó ú,

Interesting... Could you give an example of a link that has that issue? I'll look into it further.

hellishvictor commented 1 year ago

Sure, check this out: https://www.twitch.tv/videos/1676578762 On the console its ok, but on the output file is where it differs. The following using -o has errors:

0:35 | (Moderator, Verified) Moobot: Anuncios! Si quieres dejar de verlos, suscríbete, tacaño. También puedes aguantar 120 segundos y apoyar el stream <3

and here is the correct, with >:

0:35 | (Moderator, Verified) Moobot: Anuncios! Si quieres dejar de verlos, suscríbete, tacaño. También puedes aguantar 120 segundos y apoyar el stream <3

xenova commented 1 year ago

Running 

chat_downloader https://www.twitch.tv/videos/1676578762 --start_time 30 --end_time 40 -o test.txt

outputs the following to the text file (which is correct).

0:30 | (GlitchCon 2020) Noeemansilla: cristininiHYPE cristininiHYPE
0:30 | paulawolfbbys: hola chat
0:33 | (cheer 100) fabricio0311: kitinaaaaaaaaaaaa
0:34 | (Prime Gaming) mrmaxi_98xx: ole oleee
0:34 | Pablo_elElfo: cristininiNINIER cristininiNINIER
0:35 | (Moderator, Verified) Moobot: Anuncios! Si quieres dejar de verlos, suscríbete, tacaño. También puedes aguantar 120 segundos y apoyar el stream <3
0:36 | (Watching without audio) sofia2485: holaaa
0:39 | (6-Month Subscriber) agoneyrp: cristininiHOLA cristininiHOLA cristininiHOLA cristininiHOLA cristininiCORAZON
0:40 | (Prime Gaming) Ivanzz44: Hola

This might be an OS-dependent issue? What OS are you using?

hellishvictor commented 1 year ago

Here is what is happening: chat_downloader saves the output by default in UTF-8, and I use Notepad++ in spanish (and the OS as well), and it opens the chat as an ANSI file, and that's why it shows the errors. If I switch on N++ the file to UTF-8 (without bom), its shown right. Using > saves it in ANSI by default.

xenova commented 1 year ago

Gotcha! 👍

Thanks for the info.