search

xenova / chat-downloader

A simple tool used to retrieve chat messages from livestreams, videos, clips and past broadcasts. No authentication needed!
https://chat-downloader.readthedocs.io/
MIT License
958 stars 133 forks source link

[BUG] (YouTube) It will download the chat room of the latest live archive if use `channel url`. #183

Closed JC-Chung closed 2 years ago

JC-Chung commented 2 years ago

Basic information

Describe the bug

It will download the chat room of the latest live archive every 30 seconds if use channel url.

Command/Code used

If running from the command line, provide the following:

  1. The command used (including the verbose tag, -v): 
    chat_downloader -v https://www.youtube.com/channel/UCZ5Wn4ss81cVKSk74Duv2BA
  2. Output from the above command: 
    [DEBUG] Python version: 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)]
[DEBUG] Program version: 0.2.1
[DEBUG] Initialisation parameters: {'headers': None, 'cookies': None, 'proxy': None}
[DEBUG] Created YouTubeChatDownloader session.
[INFO] Site: youtube.com
[DEBUG] Program parameters: {'url': 'https://www.youtube.com/channel/UCZ5Wn4ss81cVKSk74Duv2BA', 'start_time': None, 'end_time': None, 'max_attempts': 15, 'retry_timeout': None, 'interruptible_retry': True, 'timeout': None, 'inactivity_timeout': None, 'max_messages': None, 'message_groups': ['messages'], 'message_types': None, 'output': None, 'overwrite': True, 'sort_keys': True, 'indent': 4, 'format': 'youtube', 'format_file': None, 'chat_type': 'live', 'ignore': None, 'message_receive_timeout': 0.1, 'buffer_size': 4096}
[DEBUG] Match found: "<re.Match object; span=(0, 56), match='https://www.youtube.com/channel/UCZ5Wn4ss81cVKSk7>". Running "_get_chat_by_user" function in "YouTubeChatDownloader".
[DEBUG] Chat information: {'chat': <generator object YouTubeChatDownloader._get_chat_messages_by_user_args at 0x0000023DF4541E40>, 'title': 'UCZ5Wn4ss81cVKSk74Duv2BA', 'duration': None, 'status': None, 'video_type': None, 'start_time': None, 'id': 'UCZ5Wn4ss81cVKSk74Duv2BA', '_output_writer': None, '_output_callback': None, 'format': <function ChatDownloader.get_chat.<locals>.<lambda> at 0x0000023DF459A040>, 'site': <chat_downloader.sites.youtube.YouTubeChatDownloader object at 0x0000023DF458AC70>}
[INFO] Retrieving chat for "UCZ5Wn4ss81cVKSk74Duv2BA".
[DEBUG] Starting new HTTPS connection (1): www.youtube.com:443
[DEBUG] https://www.youtube.com:443 "GET /channel/UCZ5Wn4ss81cVKSk74Duv2BA/streams HTTP/1.1" 200 None
[DEBUG] https://www.youtube.com:443 "GET /watch?v=cZOjRol77xg HTTP/1.1" 200 None
[INFO] Found a livestream: "Title" (cZOjRol77xg).
[DEBUG] Getting Live chat (Live chat replay).
[DEBUG] https://www.youtube.com:443 "GET /live_chat_replay?continuation=op2w0wRmGl5DaWtxSndvWVZVTmFOVmR1TkhOek9ERmpWa3RUYXpjMFJIVjJNa0pCRWd0aldrOXFVbTlzTnpkNFp4b1Q2cWpkdVFFTkNndGpXazlxVW05c056ZDRaeUFCTUFBJTNEQAFyAggB HTTP/1.1" 200 None
[DEBUG] Continuation parameters: {'click_tracking': {'clickTrackingParams': 'IhMIjbKGpdfb+wIVQ0wPAh1V+Qz7'}, 'continuation': 'op2w0wRmGl5DaWtxSndvWVZVTmFOVmR1TkhOek9ERmpWa3RUYXpjMFJIVjJNa0pCRWd0aldrOXFVbTlzTnpkNFp4b1Q2cWpkdVFFTkNndGpXazlxVW05c056ZDRaeUFCTUFBJTNEQAFyAggB'}
[DEBUG] Session headers: User-Agent, Accept-Language, origin, x-youtube-client-name, x-youtube-client-version, x-origin, x-goog-authuser, x-goog-visitor-id, content-type, referer
[DEBUG] Logged-in info: {'key': 'logged_in', 'value': '0'}
...
chat
...
[DEBUG] Total number of messages: 191
[DEBUG] Continuation info: {'timeUntilLastMessageMsec': 5000, 'continuation': 'op2w0wR8Gl5DaWtxSndvWVZVTmFOVmR1TkhOek9ERmpWa3RUYXpjMFJIVjJNa0pCRWd0aldrOXFVbTlzTnpkNFp4b1Q2cWpkdVFFTkNndGpXazlxVW05c056ZDRaeUFCTUFBJTNEKM6auN4NQABIA1IIIACwAQD4AQByBAgBGAF4AA%3D%3D'}
[DEBUG] Continuation info: {'continuation': 'op2w0wR8Gl5DaWtxSndvWVZVTmFOVmR1TkhOek9ERmpWa3RUYXpjMFJIVjJNa0pCRWd0aldrOXFVbTlzTnpkNFp4b1Q2cWpkdVFFTkNndGpXazlxVW05c056ZDRaeUFCTUFBJTNEKM6auN4NQABIBFIIIACwAQD4AQByBAgBGAF4AA%3D%3D'}
[DEBUG] https://www.youtube.com:443 "POST /youtubei/v1/live_chat/get_live_chat_replay?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 HTTP/1.1" 200 None
[DEBUG] Continuation parameters: {'click_tracking': {'clickTrackingParams': 'CAEQl98BIhMI0JjNpdfb-wIVMkX1BR2dHQN2'}, 'continuation': 'op2w0wR8Gl5DaWtxSndvWVZVTmFOVmR1TkhOek9ERmpWa3RUYXpjMFJIVjJNa0pCRWd0aldrOXFVbTlzTnpkNFp4b1Q2cWpkdVFFTkNndGpXazlxVW05c056ZDRaeUFCTUFBJTNEKM6auN4NQABIA1IIIACwAQD4AQByBAgBGAF4AA%3D%3D'}
[DEBUG] Session headers: User-Agent, Accept-Language, origin, x-youtube-client-name, x-youtube-client-version, x-origin, x-goog-authuser, x-goog-visitor-id, content-type, referer
[DEBUG] Logged-in info: {'key': 'logged_in', 'value': '0'}
[INFO] There are no active or upcoming livestreams with a live chat. Retrying in 30 seconds.

    If the output is too long, you can attach a text file or remove output which does not constitute to the problem.

Expected behavior

Only [INFO] There are no active or upcoming livestreams with a live chat. Retrying in 30 seconds. Instead of getting the latest live archives chat rooms.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context/information

Add any other context or information about the problem here.

xenova commented 2 years ago

Right, this is due to a recent change to YouTube where past broadcasts are shown in the "live" section. image

I'll make it so that only live or upcoming broadcasts (as you mentioned here)

xenova commented 2 years ago

https://github.com/xenova/chat-downloader/commit/0941576a4d4a43f521a30038f801980c0eb179e2 should fix this. Let me know if it works! 👍

(You will have to install the git version for now... I will make a full release soon)

upintheairsheep commented 1 year ago

Sorry for the off-topicness, but I've created that TikTok comments and replies extracter I was asking for myself. Please pull request this, as I am banned due to my issue spamming in the past, but I've just learned to program instead. https://github.com/upintheairsheep/ytdl-sheep/pull/6/files