Download (using record.sh?) in parts

[KauzDs]

When using record.sh instead of downloading the whole stream from start to finish it would be neat to be able to choose to download in parts by stream time. For example, when archiving twitch vods to Youtube, the vod can't be over 12 hours long as such when using record.sh if the stream is longer than 12 hours you need to trim it, divide it into two or more parts of 12 hours or less using ffmpeg or video editing software which can take time.

[Hakkin]

This would be pretty difficult to do flexibly in either twitchpipe or record.sh, just because of how each program works. twitchpipe has no concept of different "files", it's just dumping everything it downloads to the standard output for they next program to do whatever it wants with. One thing I've thought of before was to have some kind of "processing" option for record.sh, which would allow you to specify some command (like ffmpeg) that the .ts file will flow through instead of directly writing to the output file, but even this complicates things a lot compared to how it is now. If you want to manually hack this in, it's pretty easy, you just have to change the filename line and change the line where twitchpipe is called in record.sh to pipe into ffmpeg instead of piping to a file.

diff --git a/tools/record.sh b/tools/record.sh
index 0de8a70..b2a5760 100644
--- a/tools/record.sh
+++ b/tools/record.sh
@@ -132,9 +132,9 @@ done <<< "${IDS}"
           while :
           do
             mkdir -p "$username"
-            filename=$(printf "%s/%s.ts" $username "$(date -u '+%Y_%m_%d_%H_%M_%S_(%Z)')")
+            filename=$(printf "%s/%s_%%04d.ts" $username "$(date -u '+%Y_%m_%d_%H_%M_%S_(%Z)')")
             errf '[%s] recording to %s\n' "$username" "$filename"
-            (twitchpipe --archive --group "$GROUP" "$username" >> "$filename") 2>&1 | (
+            (twitchpipe --archive --group "$GROUP" "$username" | ffmpeg -hide_banner -loglevel warning -f mpegts -i - -c copy -segment_time 6:00:00 -f segment -reset_timestamps 1 "${filename}") 2>&1 | (
               while read -r line;
               do
                 errf '[%s] %s\n' "$username" "$line"

We add "%04d" to the filename, since ffmpeg requires this formatting for segmented output, then we pipe to the ffmpeg command ffmpeg -hide_banner -loglevel warning -f mpegts -i - -c copy -segment_time 6:00:00 -f segment -reset_timestamps 1 "${filename}" The part you care about is segment_time 6:00:00, this means it will split the file after 6 hours of video, you can change this to whatever, the formatting is hh:mm:ss.

I'm actually thinking about rewriting all the logic of twitchpipe and record.sh into a single Go program, the bash script is becoming a little unwieldy and is difficult to add complex features to. I have no idea when (or if) I will actually do this though. Moving all the logic to Go would make it much easier to add more complex features, like chat downloading or sidecar metadata files (including timed metadata like category changes, title updates, etc).

[Hakkin]

also, if you don't want to edit the record.sh script, it's simple to run the ffmpeg command on the final output file as well, just do ffmpeg -i "input_file.ts" -c copy -segment_time 6:00:00 -f segment -reset_timestamps 1 "output_file_%04d.ts" This isn't doing any re-encoding, it's simply parsing the file and copying it to the new output, so it should write the file pretty much as fast as your hard drive can support, it may still take a few minutes for very large files though.