Do You Like The Current Output of X9K3? Do You Have A Better Idea?

[futzu]

Say something.

I am open to suggestions.

./seg29.ts  start: 89731.181667 duration: 2.120000
./seg30.ts  start: 89733.301667 duration: 2.120000
./seg31.ts  start: 89735.421667 duration: 2.120000
./seg32.ts  start: 89737.541667 duration: 2.120000
#EXT-X-CUE-OUT:242.0
./seg33.ts  start: 89739.661667 duration: 2.520000
#EXT-X-CUE-OUT-CONT:2.520/242.0
./seg34.ts  start: 89742.181667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:4.640/242.0
./seg35.ts  start: 89744.301667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:6.760/242.0
./seg36.ts  start: 89746.421667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:8.880/242.0
./seg37.ts  start: 89748.541667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:11.000/242.0
./seg38.ts  start: 89750.661667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:13.120/242.0
./seg39.ts  start: 89752.781667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:15.240/242.0
./seg40.ts  start: 89754.901667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:17.360/242.0
./seg41.ts  start: 89757.021667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:19.480/242.0
./seg42.ts  start: 89759.141667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:21.600/242.0
./seg43.ts  start: 89761.261667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:23.720/242.0
./seg44.ts  start: 89763.381667 duration: 2.120000
#EXT-X-CUE-OUT-CONT:25.840/242.0
./seg45.ts  start: 89765.501667 duration: 2.120000

./seg27.ts  start: 89726.941667 duration: 2.120000
./seg28.ts  start: 89729.061667 duration: 2.120000
./seg29.ts  start: 89731.181667 duration: 2.120000
./seg30.ts  start: 89733.301667 duration: 2.120000
./seg31.ts  start: 89735.421667 duration: 2.120000
./seg32.ts  start: 89737.541667 duration: 2.120000
#EXT-X-DATERANGE:ID="1",START-DATE="2022-12-26T21:35:41.398909Z",PLANNED-DURATION=242.0,SCTE35-OUT=0xfc302f00000000000000fff01405000002967fefffe16a1ab87e014c562000010000000a00084355454900000000ebf5a2a7
./seg33.ts  start: 89739.661667 duration: 2.520000
./seg34.ts  start: 89742.181667 duration: 2.120000
./seg35.ts  start: 89744.301667 duration: 2.120000
./seg36.ts  start: 89746.421667 duration: 2.120000
./seg37.ts  start: 89748.541667 duration: 2.120000
./seg38.ts  start: 89750.661667 duration: 2.120000
./seg39.ts  start: 89752.781667 duration: 2.120000

[ashiskumarsahu]

Hey @futzu, You have made really a superb package and output are also great. I have been doing some analysis with your script and found it would be better if it can take multi resolution streams as input and output the same resolutions with master manifest.

For sidecar file inputs are in pts format. I was assuming like if it could support duration format as well. Ex, if duration is 0:01:58 - marker will get inserted here or near by segment break.

Here is another scenario, suppose we have a scte35 marker in a live stream of football match and an ad is replaced and playing with that point considering a commercial break. Now think if the match started but break is still not over but I want to trigger something that would help to back to main stream without waiting for ads break to close. i am not sure if this is possible currently or not?

Thank You

[bbgdzxng1]

In terms of producing a machine-readable structured data log to generate an as-run / as-ran report, JSON output would be cool. So that a logfile could be parsed / filtered with the super awesome jq. There's probably a way today to pass stderr to it with 2>&1 | jq [someoptions] or save stderr to logfile and parse it at a later date, but a json logfile would be JustEasy for your users. It does pose the challenge of logfile rotation, but that is common to all loggers.

But for general command-line usage at the terminal, the current stderr format is clear and concise.

System time ISO 8601 would be helpful for post log analysis of a live stream. Obviously, this going to be approximate, taking account of segment time, but it could be the program_date_time value. At present, this can be achieved from looking at file system timestamp, but since you asked for features to keep you busy, system time of when the event occurred (file creation, cue) would be helpful. Program date time in the HLS output goes some way to achieve this, but of course with a sliding window, that requires that the m3u8 is continuously logged by another mechanism.

Freestylin' here but something along the lines of:

{
  "event": {
    "id": "an_ordinal_number",
    "eventtype": "[ segment_write_out | cue ]",
    "segmentfile": "./seg146.ts",
    "cue": "/DAvAAAAAAAAAP/wFAUAAAKWf/somekindofsctenonsense==",
    "start": "1147.466667",
    "duration": "6.000000",
    "program_timestamp": "2022-12-27T19:06:15Z",
    "pts": "89981.281522"
  },
  etc... presence of cue and segmentfile would depend on eventtype.
}

Edit: In hindsight, extension of the m3ufu json format to support x9k3 events would glue the suite together rather nicely. Taking another look at the json format of m3ufu, it has all in the info there - it would just be preferable to not have to write and read it out from an intermediary m3u8 on filesystem - logging of as-run from the core worker of x93k would be cleaner.

Also, write-out of filenames inmycustomstring_%06d ordinal or ISO 8601 strftime() systemtime are something that other segmenters tend to do. Again, only if you are looking for work / ideas.

Full disclosure: I'm not actively using x9k3 in production, I use AWS MediaLive, but this is a super interesting project that I keep an eye on - and an open source tool is much needed tool for the industry. I think the only other open-source one of note is Shaka Packager, which is very limited when it comes to live.

I like your work.

[futzu]

Hey @futzu, You have made really a superb package and output are also great. I have been doing some analysis with your script and found it would be better if it can take multi resolution streams as input and output the same resolutions with master manifest.

For sidecar file inputs are in pts format. I was assuming like if it could support duration format as well. Ex, if duration is 0:01:58 - marker will get inserted here or near by segment break.

Here is another scenario, suppose we have a scte35 marker in a live stream of football match and an ad is replaced and playing with that point considering a commercial break. Now think if the match started but break is still not over but I want to trigger something that would help to back to main stream without waiting for ads break to close. i am not sure if this is possible currently or not?

Thank You

I have been thinking about what you were wanting to do with ABR, and how to do it. I am working on it.

I was wondering if duration might be something people wanted, I usually call that "hls time", I dont what the official name is, So you would prefer duration over pts?

Interrupting commercial is definitely possible, In live mode with a sidecar file you can add pts, cue lines while its running and, once x9k3 reads them it erases from the file.

I do it like this

printf '114.633333,/DAhAAAAAAAAAP/wEAUAAAAJf78A/gASZvAACQAAAACokv3z\n' >> sidecar.txt

It should interrupt , but if it doesn't it would not be hard to make it do so.

[futzu]

@bbgdzxng1 I have been wanting to unify m3ufu and x9k3, and what you're saying makes a lot of sense. I did a project recently where I was taking in m3u8 files as input for x9k3, so I used m3ufu to handle it and it worked surprisingly well.

I was wondering when someone was going to bring up segment naming :) I expected it much sooner to be honest. My approach has been not to add features until someone requests them, so I don't add a bunch of stuff nobody uses. I will take this as a request.

I am trying to figure out how to do ABR effectively.

I am not offended you not using x9k3, if it doesnt give you what you need, but I dont trust amazon at all myself :)

[bbgdzxng1]

One of the challenges of dealing with ABR is keeping the segments in sync for fussy players or where scene change i-frames may not always have been inserted by x264 in lower-res profiles. I try to keep a consistent IDR keyframe heartbeat in the encode, irrelevant on any I-frames that a codec may or may not throw in there. And when generating the ABR with FFmpeg, the tee muxer does help create discreet files that have some commonality, or FFmpeg's var_stream_map for segmented output. There are a tonne of naive FFmpeg ABR scripts out there that do produce multi-bitrate, but have no appreciation that PTS commonality between renditions has a huge value for downstream packagers or players.

One option may be to feed your packager with a multiplexed A/V stream, containing all the video and audio ~segments~ streams in one TS bundle and demux that, but it does rather tie your users to an encoder that can supply multiplexed A/V. It ain't easy.

Obviously, there is already bento, GPAC/mp4box and Apple Segmenter which can operate as pure packagers, and Shaka ~Player~ Packager that packages and does some basic cue-insertion, albeit in a limited way. They each have different ways of demuxing, segmenting and packaging an ABR input, so maybe there is some inspiration that you can gather from these.

My requirements for packaging are now a lot more complex these days - I've been in a simple HLS shop, but now deal with DASH, multi-flavor DRM, even some CMAF... all with ad insertion. But I'm so pleased that someone is leading the charge on real-time HLS segmentation with all the various flavors of cue messages, SCTE data tracks, HLS manifest decoration, so many flavors of CUE-OUT etc in plain old HLS. You are filling a huge gap in the market for the open-source HLS w/ad marker. With FFmpeg & x9k3 the barrier to creating a linear TV channel with ad insertion no longer relies on trying to do the manifest manipulation on origin with Nginx and a dirty script.

As for the file naming, please don't develop on my account, but I do know from experience that a clean file naming convention really makes debugging so much easier, so I am confident that it will be valued by your users. strftime() named files make Catchup VoD of a live channel just a matter of manifest index creation between timestamped files. Some countries' regulatory bodies require by law that a N-week window is kept on record in case of complaint, even if not doing Catchup VoD. strftime() makes that whole process easy to achieve and housekeep. But it sounds like you already had already spotted that one.

I still rely on good ole threefive for my analysis (I should really be using cuei) and m3ufu is, well, TotallyFreakinAwesome. As ever, I am very grateful that you and your suite exists.

[ashiskumarsahu]

time

Hey @futzu, You have made really a superb package and output are also great. I have been doing some analysis with your script and found it would be better if it can take multi resolution streams as input and output the same resolutions with master manifest. For sidecar file inputs are in pts format. I was assuming like if it could support duration format as well. Ex, if duration is 0:01:58 - marker will get inserted here or near by segment break. Here is another scenario, suppose we have a scte35 marker in a live stream of football match and an ad is replaced and playing with that point considering a commercial break. Now think if the match started but break is still not over but I want to trigger something that would help to back to main stream without waiting for ads break to close. i am not sure if this is possible currently or not? Thank You

I have been thinking about what you were wanting to do with ABR, and how to do it. I am working on it.

I was wondering if duration might be something people wanted, I usually call that "hls time", I dont what the official name is, So you would prefer duration over pts?

Interrupting commercial is definitely possible, In live mode with a sidecar file you can add pts, cue lines while its running and, once x9k3 reads them it erases from the file.

I do it like this

printf '114.633333,/DAhAAAAAAAAAP/wEAUAAAAJf78A/gASZvAACQAAAACokv3z\n' >> sidecar.txt

It should interrupt , but if it doesn't it would not be hard to make it do so.

Hey @futzu , Thanks for replying.

For ABR how you are planing? As x9k3 takes mpegts input, is it possible to input multiple mpegts? Do you know how ffmpeg encode when passed a m3u8 stream as input? Ex: ffmpeg -i https://vidcdn.vidgyor.com/news24-origin/liveabr/playlist.m3u8 -c:v h264 -c:a aac -hls_time 1 -f mpegts -

input http stream having a master manifest with 4 media manifest, so each version of segments should be simultaneously read and insert markers on each version sententiously.

printf '114.633333,/DAhAAAAAAAAAP/wEAUAAAAJf78A/gASZvAACQAAAACokv3z\n' >> sidecar.txt

Is the pts_time same with duration in seconds? Suppose I want to add a splice point at 2 minutes of content. Can I pass 120.000000 in above command?

Thank you

[futzu]

@bbgdzxng1 I am term iframe as a general term, I am including IDR, I check for iframes 4 different ways. Ive been thinking the same thing, just encode the variants to mpegts, and then start a thread to segment each variant with x9k3. The only thing stopping me right now is parsing out the codecs for the master.m3u8, I can parse SPS for h264, but that is the only one I can do so far. I just need to read up. Another option would be to use the -hls_flags single_file and segment those with x9k3

[futzu]

@ashiskumarsahu that is what Im thinking the multiple mpegts inputs, because with m3u8 files, x9k3 would have to assemble the segments and slice them back up. I wont have that any time soon though.

114.633333 is a pts time, x9k3 doesnt currently duration times, PTS is stamped on the mpegts stream, it goes from 0 to 95443.717678 and then rolls over. depending on how you encode it two minutes could be at 120 , or it could be at 65789.111111.

Everything is pts because SCTE-35 is always in terms of pts.

[futzu]

@bbgdzxng1 have you tried gumd? That is my favorite.

[bbgdzxng1]

@futzu my days of UDP multicast are over, but I see what you are doing with gumd. I'm either using pipes between apps or srt.

I can parse SPS for h264, but that is the only one I can do so far. I just need to read up.

As for getting the CODECS, I work in higher-level scripts and commands, rather than the elegance of python, so I use FFprobe, which would be far too heavy for your world... and not particularly robust.

$ ffprobe -hide_banner -loglevel error -i "${infile}" -select_streams v:0 -show_entries stream=codec_tag_string -print_format default=nokey=1:noprint_wrappers=1
$ printf "%02x" $(ffprobe -hide_banner -loglevel error -i "${infile}" -select_streams v:0 -show_entries stream=level -print_format default=nokey=1:noprint_wrappers=1) # Get the level

If you have not played with it yet, take a look at the quite brilliant https://github.com/wader/fq, which is a whole header parser written in go (json & hexdump outputs). The gentleman who has put fq together has "MIME codec encode/decoder "avc1.PPCCLL" on his TODO list.

But I suspect that you'll prefer to pull it out of the SPS yourself, but no harm in stubbing something out in FFprobe or fq.

Here's the genius of 'fq' of how easy it is to navigate the SPS...

$ ffmpeg -hide_banner -loglevel error -f lavfi -i testsrc -c:v libx264 -frames:v 1 -bsf:v 'h264_mp4toannexb' -f h264 - \
  | fq --decode 'avc_annexb' '[ .[] | objects | select(.nal_unit_type=="sps") | .sps.level_idc | . ]'
[
  "1.3"
]

or

$ ffmpeg -hide_banner -loglevel error -f lavfi -i testsrc -c:v libx264 -frames:v 1 -bsf:v 'h264_mp4toannexb' -f h264 - \
  | fq --decode 'avc_annexb' ' .[] | objects | select(.nal_unit_type=="sps") | display'

      |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[1]{}: nalu (avc_nalu)
      |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|  sps{}: (avc_sps)
  0x00|f4                                                         |.                   |    profile_idc: "high_444_predictive_profile" (244)
  0x00|   00                                                      | .                  |    constraint_set0_flag: false
  0x00|   00                                                      | .                  |    constraint_set1_flag: false
  0x00|   00                                                      | .                  |    constraint_set2_flag: false
  0x00|   00                                                      | .                  |    constraint_set3_flag: false
  0x00|   00                                                      | .                  |    constraint_set4_flag: false
  0x00|   00                                                      | .                  |    constraint_set5_flag: false
  0x00|   00                                                      | .                  |    reserved_zero_2bits: 0
  0x00|      0d                                                   |  .                 |    level_idc: "1.3" (13)
  0x00|         91                                                |   .                |    seq_parameter_set_id: 0
  0x00|         91                                                |   .                |    chroma_format_idc: "4:4:4" (3)
  0x00|         91                                                |   .                |    separate_colour_plane_flag: false
  0x00|         91                                                |   .                |    bit_depth_luma: 8
  0x00|            9b                                             |    .               |    bit_depth_chroma: 8
  0x00|            9b                                             |    .               |    qpprime_y_zero_transform_bypass_flag: false
  0x00|            9b                                             |    .               |    seq_scaling_matrix_present_flag: false
  0x00|            9b                                             |    .               |    log2_max_frame_num: 4
  0x00|            9b                                             |    .               |    pic_order_cnt_type: 0
  0x00|            9b                                             |    .               |    log2_max_pic_order_cnt_lsb: 6
  0x00|               28                                          |     (              |    max_num_ref_frames: 4
  0x00|               28                                          |     (              |    gaps_in_frame_num_value_allowed_flag: false
  0x00|               28 28                                       |     ((             |    pic_width_in_mbs: 20
  0x00|                  28 3f                                    |      (?            |    pic_height_in_map_units: 15
  0x00|                     3f                                    |       ?            |    frame_mbs_only_flag: true
  0x00|                     3f                                    |       ?            |    direct_8x8_inference_flag: true
  0x00|                        60                                 |        `           |    frame_cropping_flag: false
  0x00|                        60                                 |        `           |    vui_parameters_present_flag: true

fq is written in go, so it may help you stub something out quite quickly - although I know you are perfectly capable of doing this all yourself in python. There are already header parsers for avc_annexb / hevc_annexb, which make inspecting stuff like SPS really quick. Not sure whether there is AAC and all the possible HLS codecs yet, but you can see the formats that are quickly navigable with $ fq -h formats. I think you'll like 'fq', even if it is overkill for the immediate task. But it is the coolest thing in easy media inspection/navigation that I have seen for quite a while.

Dirty workaround: The CODECS statement is only really a constraint so that a HLS player can check whether it should skip that rendition or not in case of mixed 264/265 in the same master, so lazy folks have been known to hardcode something like avc1/high/4.1 and HE-AACv1, just to get going. Since avc high, by definition, is a superset of main & baseline and HE-AACv1 is a superset of AAC-LC. It is obviously not good to hardcode, but you could stub it out for the time being (don't shoot me for what is a knowingly nasty workaround). The CODECS is basically just a capabilities check that the player uses at master-parse-time to avoid subsequently decoding a stream it would be incapable of playing.

[ashiskumarsahu]

@ashiskumarsahu that is what Im thinking the multiple mpegts inputs, because with m3u8 files, x9k3 would have to assemble the segments and slice them back up. I wont have that any time soon though.

114.633333 is a pts time, x9k3 doesnt currently duration times, PTS is stamped on the mpegts stream, it goes from 0 to 95443.717678 and then rolls over. depending on how you encode it two minutes could be at 120 , or it could be at 65789.111111.

Everything is pts because SCTE-35 is always in terms of pts.

Hey @futzu , That cool. Let me try something from my end to support input/output multiple streams.

Is there any easiest way to get pts time in live mode for SCTE insertion? or I should use date range? If yes please share any documentation which can clarify how to use date rang, I mean does it take any utf time input? or in overall how does it work?

Thank You

[futzu]

@ashiskumarsahu I know PTS is a pain in the ass, but that's the way SCTE-35 works. What's even worse, MPEGTS has four different timestamps embedded in the stream, that gets really confusing.

in v.0.1.75 the start in the output is PTS.

./seg0.ts   start: 102.506478   duration: 2.126855
./seg1.ts   start: 104.633333   duration: 2.000000
./seg2.ts   start: 106.633333   duration: 2.000000
./seg3.ts   start: 108.633333   duration: 2.000000

I just pushed version 0.1.77 The start and end in the output are PTS timestamps.

a@debian:~/build/clean/x9k3$ pypy3 x9k3.py -i ../la-slim.ts -s sidecar.txt 
./seg0.ts:  start:102.506478    end:104.633333  duration:2.126855
./seg1.ts:  start:104.633333    end:106.633333  duration:2.0
./seg2.ts:  start:106.633333    end:108.633333  duration:2.0
./seg3.ts:  start:108.633333    end:110.633333  duration:2.0
./seg4.ts:  start:110.633333    end:112.633333  duration:2.0
./seg5.ts:  start:112.633333    end:114.633333  duration:2.0
./seg6.ts:  start:114.633333    end:114.7   duration:0.066667

[futzu]

@bbgdzxng1, I'm trying to avoid doing a popen call with an external binary, that rarely ages well. However, that fq looks pretty cool and I am not above blatantly copying somebody's code, that how I did the CRC stuff in threefive. :)

[futzu]

@bbgdzxng1 I am looking for a SRT lib I can use with new_reader.

[bbgdzxng1]

I am looking for a SRT lib I can use with new_reader.

I've found the Haivison reference tools & examples to be pretty good, even if they are tagged as "not for production"... but I believe they are in C++. https://github.com/Haivision/srt/blob/master/apps/srt-live-transmit.cpp, but again, I've only used them as compiled binaries. I've used FFmpeg and TSDuck, but Haivision live-transmit tended to be more reliable. I've not played with any SRT libraries in the pythonic domain.

fq is very cool... and I could imagine a great crossover between your toolset and fq. I had wondered how far different cuei's Go parser is from a scte-35 parser forfq, if and when fq gets some momentum behind it. But it is a seriously cool tool for parsing and navigating many different formats and headers, all presented in a common navigable tree which is a godsend for fast header navigation and inspection. The kind developer has just added a branch for some basic EIA-608 SEI extraction (which I know you dealt with that beast back in your scc2vtt days).

[futzu]

@ashiskumarsahu what you want I can do for you in about 2 weeks, if you want to hire me. As far as me adding that to x9k3, I may eventually do it, but it would be months before I even start on it. I Work on x9k3 when I have some spare time. Its not my focus. I did add setting the insert time to 0 for splice immediate in sidecar files in v0.1.81 https://github.com/futzu/x9k3#release-notes

[ashiskumarsahu]

@futzu Please share your your email. So that I can share my requirement.

Thank You

[ashiskumarsahu]

@ashiskumarsahu what you want I can do for you in about 2 weeks, if you want to hire me. As far as me adding that to x9k3, I may eventually do it, but it would be months before I even start on it. I Work on x9k3 when I have some spare time. Its not my focus. I did add setting the insert time to 0 for splice immediate in sidecar files in v0.1.81 https://github.com/futzu/x9k3#release-notes

I want to hire you for that project. Could you please share your email? I will send my requirement and in reply I need a quote. Accordingly we can make the deal.