chat log download and reformatting for youtube

[NotQuiteApex]

Not a majorly important change, but pulling chat logs along with stream vods will help with archiving.

In addition, the ability to upload VODs with the proper sections of chat ripped and uploaded to YouTube using their advanced closed captioning system would be perfect. More research will need to be done with this before it can even be ready to implement, and perhaps could be split into its own issue.

[NotQuiteApex]

This seems like the proper resource on the matter, and using the advanced SAMI or RealText formats might be what vodbot needs as positioning is not important. These would be very simple scripts to implement as it's simply taking everything line by line and pumping it through a reformatter function.

https://support.google.com/youtube/answer/2734698?hl=en

[NotQuiteApex]

Twitch appears to have an undocumented endpoint for grabbing "comments" from VODs. The first query should be of the video's id and the offset from the start of the slice, in seconds. The returned data will have a pagination object to use for subsequent queries. Simple enough.

"https://api.twitch.tv/v5/videos/{video_id}/comments?content_offset_seconds={start_of_video}" "https://api.twitch.tv/v5/videos/{video_id}/comments?cursor={cursor}"

There may be a GQL query to use instead, which would be preferable because of how VodBot is currently set up; independent of any of the API endpoints provided by Twitch.

[NotQuiteApex]

After some research, it appears the GQL query will revolve around "videoComments". Formatted something like this:

GET_VIDEO_COMMENTS_QUERY = """
{{ videoComments(videoID: "{video_id}") {{
        VideoCommentConnection
}}  }}
"""

I've yet to actually test this, I will when possible. The main issue is it's unclear how pagination will work, potentially similar to GET_CHANNEL_VIDEOS_QUERY?

[NotQuiteApex]

# IRC Chat query
GET_VIDEO_COMMENTS_QUERY = """
{{ videoComments(videoID: "{video_id}", first: {first}, after: "") {{
        pageInfo {{ hasNextPage }}
        edges {{ cursor
            node {{
                commenter {{ id login displayName }}
                message {{ userColor fragments {{ mention {{ id login displayName }} text }} }}
                contentOffsetSeconds
                createdAt id state
            }}
        }}
}}  }}
"""

Here is the query vodbot will likely use, properly annotated with requested info. The most useful bits are commenter, contentOffsetSeconds, and message. The state field will need to be reviewed to ensure that certain messages like those deleted are not exported, along with comments pending review. Or perhaps a config filter?

[NotQuiteApex]

Currently the server returns an unknown error when using this query. It may be out of date or I'm formatting something horribly wrong.

[NotQuiteApex]

It appears that this form of grabbing comments has been deprecated, as the server just does not want to respond to any form of request. However, I discovered that the video query can pull comments!

# IRC Chat query
GET_VIDEO_COMMENTS_QUERY = """
{{ video(id: "{video_id}") {{
    comments(contentOffsetSeconds: 0) {{
        pageInfo {{ hasNextPage }}
    }}
}} }}
"""

The above is a test example, with the id being one of my recent streams. I will post the full query once I get a working example.

[NotQuiteApex]

# IRC Chat query
GET_VIDEO_COMMENTS_QUERY = """
{{ video(id: "{video_id}") {{
    comments(contentOffsetSeconds: 0) {{
        pageInfo {{ hasNextPage }}
        edges {{ cursor
            node {{
                contentOffsetSeconds state
                commenter {{ id login displayName }}
                message {{
                    userColor 
                    fragments {{ mention {{ id login displayName }} text }}
}}  }}  }}  }}  }} }}
"""

Here's the Twitch GQL query for getting chat logs or comments from videos. It's rather messy, as you can see, but it requires no authenticating and it works for all videos. I'll get to work on a new branch soon.