Batch service

[camjac251]

I'm very intrigued by the readme's description of the archive function. All of the "archiving" scripts out there for live twitch streams work with the API and a schedule/cron system for checking whether or not someone goes live, then starts the recording from that point with streamlink/youtube-dl. I'm using https://github.com/jrudess/streamdvr which uses a schedule system but with the way it operates, up to 60 seconds of the beginning could be cut off (with a 60 second cron schedule).

Could this script do something similar for multiple streamers? streamdvr records to username_site_YYYYmmdd_timestarted.ts using streamlink then uses ffmpeg to fix the aac bitstream afterward and conforms it to an .mp4 from capturing/TWITCH/streamer/username_site_YYYYmmdd_timestarted.ts to captured/TWITCH/streamer/username_site_YYYYmmdd_timestarted.ts. This works as a daemon and works on all of the added streamers to config.

[Hakkin]

Could this script do something similar for multiple streamers?

Yes, the script in tools/record.sh is already able to do this, you just pass multiple usernames to it and it will monitor and record all their streams. Rather than using polling, we connect to Twitch's websocket server and receive push notifications for when streams start, this is typically nearly instant, so very little or none of the start of the stream is cut off.

The script records to username/%Y_%m_%d_%H_%M_%S_(%Z).ts, there is an open pull request for allowing more customizable naming, but work has seemingly stopped on it. The script has an option to print the filename when the recording ends, -p, this can be chained with other scripts to do post-processing, like converting to MP4 or moving/renaming the file.

[camjac251]

I should have looked closer before opening the issue. Aren't websockets cached on the Twitch API? I've heard from others that sometimes it can take up to 2-3 minutes for a response for when someone goes live. I haven't seen this archive functionality in streamlink or youtube-dl. I get about 35 seconds of buffer on running it for 3 seconds.

I was thinking that this tool might've used the VOD system to capture the beginning part of the stream.

[Hakkin]

I believe you're thinking of webhooks, which are different than websockets, those indeed can be delayed by up to a few minutes. The websocket is Twitch's "PubSub" endpoint, which is typically used for things like monitoring realtime events such as bit donations, channel subscriptions, things like that. We're listening for an internal, undocumented event that Twitch uses to signal the start of a stream, it's used to refresh the player if someone is already on the streamer's profile when the stream starts, so it's typically sent instantly as the stream starts.

The "archive" option on the main twitchpipe program simply starts downloading from the oldest visible segment in the HLS playlist when it connects, rather than the newest, so basically preferring "archival" over "latency", it doesn't necessarily mean it will download everything from before the program was ran, just whatever segments are still visible in the HLS playlist.

[camjac251]

I thought websockets would've been impossible for Twitch. All of the useful stuff seems undocumented. I asked in a Twitch API discord server and was told that only webhooks existed for live channels and that Pubsub was deprecated.