search

xenova / chat-downloader

A simple tool used to retrieve chat messages from livestreams, videos, clips and past broadcasts. No authentication needed!
https://chat-downloader.readthedocs.io/
MIT License
948 stars 132 forks source link

Live Chat Replay #35

Closed AaronTrotter closed 3 years ago

AaronTrotter commented 3 years ago

How can I be sure youtube live chat replay is returning the all the comments and not just the top chat?

It definitely is excluding a lot of them.

xenova commented 3 years ago

Can you please give an example? (The command you used including the YouTube video URL)

The tool should display everything which appears on the chat replay, so if you can find a message which is displayed on the website, but not retrieved, I'll be sure to look into it further.

xenova commented 3 years ago

Are you sure there are 1.5 million messages?

The video is 23 minutes long, with only 400k views. If there are 1.5 million messages, that would mean ~11k per second, which I highly doubt is correct.

If you check out the chat replay, the number of messages seem to correspond with what the tool is getting (~8 per second)

Also, it would seem as though the first message is sent at 2:21: Rayson Dunn: ​1, meaning the above estimates would be even more absurd.

AaronTrotter commented 3 years ago

This is what YT analytics is telling me.

The youtube cap is 2000 live chat messages per second and the aim of this video was an interactive experience which required people to constantly type into the chat

Per-minute that would be 120k if we reached the maximum limit which we did on a few occasions. Youtube would frequently say "Network busy" and prevent users from typing. The game interacted with youtube via the YT Live API which was also limited to 2000 per second and i can confirm we had hit this in-game. The reason we want to use your tool is because the live video no longer has a live chat id and so we are looking at other means of extracting the names of players.

Perhaps this is something you would be interested in helping us on and we can take this off git and into email? Perhaps we could hire you for a few hours to take a look to see if this is possible?

xenova commented 3 years ago

Oh that is quite strange indeed. The tool works by loading the video page and simulating http requests for the next "continuation" of the live chat replay. So, the tool will retrieve everything to be shown by the chat replay. In other words, the tool will only ever show what is visible on the YouTube video page.

I suppose YouTube implemented a cap when showing messages from a past live stream (possibly for a case such like this) to improve the viewer's experience (i.e. not loading 1.5 million messages). The tool does allow for passing a cookies file to the program, but I'm unsure whether this will fix anything.

A simple google search for missing YouTube livestream messages does not seem to yield any results, so it must be a "feature" they added in without documenting.

Excuse me for not knowing, but on the "uploader's side", can you see all the messages? Do they provide an option for downloading everything? Is there a similar UI for seeing all the messages? I understand that YouTube says that 1.5 million messages were sent, but do they provide a means of accessing it?

If you want, you can send me an email at admin@xenova.com for us to discuss further - this is quite intriguing!