search

futzu / SCTE-35_SuperKabuki

SCTE-35 Inserter for MPEGTS. SuperKabuki is SCTE-35 Packet Injection for Ad Insertion, powered by threefive.
https://github.com/futzu/SuperKabuki
BSD 2-Clause "Simplified" License
12 stars 2 forks source link

Unable to insert SCTE-35 in TV stream #7

Closed Avencheng closed 4 months ago

Avencheng commented 4 months ago

I think it's better to create a new topic to discuss this issue that i've mentioned before.

I have a TV stream as my input, but SCTE35 was never inserted. Below is the outputs of my superkabuki command :

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Output File:    output.ts
PMT Section Length: 55
Program Number: 671
PCR PID: 6119
Program Info Length: 12

Added Registration Descriptor:
    b'\x05\x04CUEI'

Found Streams:
    Stream Type: 27  PID: 6116  EI Len:  20
    Stream Type: 3  PID: 6117  EI Len:  0

Added Stream:
    Stream Type: 134 PID: 134 EI Len:  0

I can see the PID PCR are much bigger from the output of threefive show of the TV stream, :

(ffmpeg7_kabuki59) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show udp://@229.0.0.7:9000

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Program: 671
    Service:    CTINews-HD              
    Provider:   DiviCom Customer
    Pid:        6115
    Pcr Pid:    6119
    Streams:
                    Pid: 6116[0x17e4]   Type: 0x1b AVC Video
                    Pid: 6117[0x17e5]   Type: 0x3 MP2 Audio

Even if i used my ffmpeg delay command to delay the TV stream and use its output as the input for superkabuki command, SCTE35 was still never inserted.

The ffmpeg delay command: ffmpeg -re -i 'udp://@229.0.0.7:9000?overrun_nonfatal=1&fifo_size=50000000' -filter_complex "[0:v]tpad=start_duration=2[v];[0:a]adelay=2s:all=true[a]" -map "[v]" -map "[a]" -vcodec libx264 -f mpegts 'udp://229.0.0.7:9001?pkt_size=1316'

The output of threefive show of the delay stream:

(ffmpeg7_kabuki59) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show udp://@229.0.0.7:9001

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Program: 1
    Service:    Service01
    Provider:   FFmpeg
    Pid:        4096
    Pcr Pid:    256
    Streams:
                    Pid: 256[0x100] Type: 0x1b AVC Video
                    Pid: 257[0x101] Type: 0x3 MP2 Audio

How can I deal with this issue? Thanks.

futzu commented 4 months ago

I have no idea,I haven't seen a single line of your code or the video stream. I don't know what you're trying to do with your "delay" stream, that makes no sense to me. This is not a super kabuki issue.

I have no problems.

I am running

a@slow:~$ gums -i trim.ts

        Multicast Stream
        udp://@235.35.3.5:3535

        Source
        0.0.0.0:42369

        5009.78 MB sent in 125.04 seconds. 40.07 MB/Sec

in another terminal I ran 

a@slow:~$ superkabuki -i udp://@235.35.3.5:3535 -s sidecar.txt  -p 777  -o fu47.ts

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Output File:    fu47.ts
PMT Section Length: 29
Program Number: 1
PCR PID: 257
Program Info Length: 0

Added Registration Descriptor:
        b'\x05\x04CUEI'

Found Streams:
        Stream Type: 15  PID: 256  EI Len:  6
        Stream Type: 27  PID: 257  EI Len:  0

Added Stream:
        Stream Type: 134 PID: 777 EI Len:  0

Inserted Cue:
        @1.466733, /DAgAAAAAAAAAP/wDwUAADA5f//+AFJlwDA5AAAAAHnzlIM=

....

Before:

a@slow:~$ threefive show trim.ts
Read 100 packets

Program: 1

        Service: Service01
        Provider: FFmpeg
        Pid:      4096
        Pcr Pid:  257
        Streams:
                  Pid           Type
                  256 [0x100]   0xf     AAC Audio
                  257 [0x101]   0x1b    AVC Video

After:

a@slow:~$ threefive show fu47.ts
Read 100 packets

Program: 1

        Service: 
        Provider: 
        Pid:      4096
        Pcr Pid:  257
        Streams:
                  Pid           Type
                  256 [0x100]   0xf     AAC Audio
                  257 [0x101]   0x1b    AVC Video
                  777 [0x309]   0x86    SCTE35 Data  <----- SCTE-35 Added
futzu commented 4 months ago

Stop thinking, stop writing code, and listen to what I'm saying and do it like I'm saying.

"I can see the PID PCR are much bigger from the output of threefive show of the TV stream, "

That's because you re-encoded the stream.

Keep it simple. Here's a one liner. I am guessing you're a Windows guy, so may not be real familiar with pipes. Pupes are heavily used on UNIX based systems like linux. 

ffmpeg -copyts -re -i 'udp://@229.0.0.7:9000?overrun_nonfatal=1&fifo_size=50000000' -map a -map v -c copy -f mpegts - | superkabuki -s sidecar.txt -p 999 -o output.ts

This sends the output of ffmpeg directly to superkabuki via stdout. Both ffmpeg and SuperKabuki write messages to stderr.

if you want to sendt the superkabuki output via multicast , just pipe it again to gums like this:

ffmpeg -copyts -re -i 'udp://@229.0.0.7:9000?overrun_nonfatal=1&fifo_size=50000000'  \
-map a -map v -c copy -f mpegts - | superkabuki -s sidecar.txt -p 999  \
| gums -a 235.35.3.5:3535

That's it.

Avencheng commented 4 months ago

OMG! i tried your commands, it worked! Thanks a lot!

Sorry for didn't explain clearly about the delay stream. Because I have another process doing the detection on udp://229.0.0.7:9000 and also making the sidecar. In order to keep the insert_pts bigger than video PTS, I need to delay the udp://229.0.0.7:9000 for superkabuki. Please refer the diagram below. diagram

I think the ffmpeg has basic startup time, so the ffmpeg command you provided can still "delay" the stream. I might have to find another way to delay stream if I need more time for the detection, because it will show the error of Filtering and streamcopy cannot be used together if i use -filter_complex and -c copy together.

But there is still something confuses me :

First, the TV stream udp://229.0.0.7:9000 is from the IRD（Integrated Receiver Decoder).

If i use ffmpeg -i udp://229.0.0.7:9000 -c copy IRD_ffmpeg.ts to record it as a ts file and using it as my program input source, everything worked find. The output of threefive show of IRD_ffmpeg.ts

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show IRD_ffmpeg.ts

Program: 1
    Service: Service01
    Provider: FFmpeg
    Pid:  4096
    Pcr Pid: 256
    Streams:
     Pid: 256[0x100] Type: 0x1b AVC Video
     Pid: 257[0x101] Type: 0x3 MP2 Audio

superkabuki can insert cues successfully

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ superkabuki -i IRD_ffmpeg.ts -s sidecar_IRD_ffmpeg.txt -p 999 -o IRD_ffmpeg_output.ts

Output File:    IRD_ffmpeg_output.ts
PMT Section Length: 23
Program Number: 1
PCR PID: 256
Program Info Length: 0

Added Registration Descriptor:
        b'\x05\x04CUEI'

Found Streams:
        Stream Type: 27  PID: 256  EI Len:  0
        Stream Type: 3  PID: 257  EI Len:  0

Added Stream:
        Stream Type: 134 PID: 999 EI Len:  0

Inserted Cue:
        @302.358733, /DAtAAAAAAAAAP/wFAUAAAABf+/+AZ8iiP4AUxWIAAABAAAIAQZDVUVJKB+lG4fL

Inserted Cue:
        @422.478733, /DAoAAAAAAAAAP/wDwUAAAABf0/+AkQYOAAAAQAACAEGQ1VFSSgfGGnDMQ==
...

The output of threefive show of IRD_ffmpeg_output.ts

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show IRD_ffmpeg_output.ts

Program: 1
    Service: Service01
    Provider: FFmpeg
    Pid:  4096
    Pcr Pid: 256
    Streams:
                         Pid: 256[0x100] Type: 0x1b AVC Video
                         Pid: 257[0x101] Type: 0x3 MP2 Audio
                         Pid: 999[0x3e7] Type: 0x86 SCTE35 Data

But if i use TSDuck command tsp -I ip 229.0.0.7:9000 -O file IRD_TSDuck.ts to record it. The output of threefive show of IRD_TSDuck.ts will be :

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show IRD_TSDuck.ts

Program: 671
    Service: CTINews-HD              
    Provider: DiviCom Customer
    Pid:  6115
    Pcr Pid: 6119
    Streams:
                Pid: 6116[0x17e4] Type: 0x1b AVC Video
                Pid: 6117[0x17e5] Type: 0x3 MP2 Audio

Which is the same as the output of threefive show for udp://229.0.0.7:9000.

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show udp://@229.0.0.7:9000

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Program: 671
    Service: CTINews-HD              
    Provider: DiviCom Customer
    Pid:  6115
    Pcr Pid: 6119
    Streams:
             Pid: 6116[0x17e4] Type: 0x1b AVC Video
             Pid: 6117[0x17e5] Type: 0x3 MP2 Audio

But cues never inserted

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ superkabuki -i IRD_TSDuck.ts -s sidecar_IRD_TSDuck.txt -p 999 -o IRD_TSDuck_output.ts

Output File:    IRD_TSDuck_output.ts
PMT Section Length: 55
Program Number: 671
PCR PID: 6119
Program Info Length: 12

Added Registration Descriptor:
        b'\x05\x04CUEI'

Found Streams:
        Stream Type: 27  PID: 6116  EI Len:  20
        Stream Type: 3  PID: 6117  EI Len:  0

Added Stream:
        Stream Type: 134 PID: 999 EI Len:  0

Only the SCTE35 stream added successfully, the threefive show output of IRD_TSDuck_output.ts

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show IRD_TSDuck_output.ts

Program: 1
    Service: 
    Provider: 
    Pid:  6115
    Pcr Pid: 6119
    Streams:
                                         Pid: 999[0x3e7]   Type: 0x86    SCTE35 Data      
                                         Pid: 6116[0x17e4]     Type: 0x1b AVC Video
                                         Pid: 6117[0x17e5]    Type: 0x3 MP2 Audio

The same situation if i use udp://229.0.0.7:9000 as superkabuki's input, cues ban not be inserted.

In conclusion, FFmpeg seems to automatically change stream's PID and remove something like CAT, NIT, EIT etc. And superkabuki can only insert cues in the stream output by ffmpeg ?

if you want to take a look on IRD_TSDuck.ts or IRD_ffmpeg.ts, let me know how would you like me to send it to you.

Or let me know what information you want me to show you here. Thanks!

futzu commented 4 months ago

You're mixing up a lot of things here, One thing you've been doing wrong is not copy over the timestamps, it will never work if you don't do that. You don't need the delay stream, but if if works for you okay. Sending yourself multicast streams on the same server is not a good idea, multicast is about ten times slower than working on local files.

I am 100% certain that the problem is on your end and not superkabuki.

Keep it simple and you're a smart guy, you'll figure it out.

futzu commented 4 months ago

Are you re-encoding after superkabuki inserts the SCTE-35? Because if you don't explicitly map the SCTE-35 stream ffmpeg will drop it. If you do map it , they will still change the type to 0x6, which is fine, if you're using my stuff, most SCTE-35 parsers won't parse it. That's not superkabuki, it's ffmpeg. ffmpeg does that with all SCTE-35.

Avencheng commented 4 months ago

Hi men, thanks for your reply!

No, I don't re-encode the stream after superkabuki inserts SCTE-35. All I have to do is insert the SCTE-35 in udp://229.0.0.7:900 and send it out, so there is no any ffmpeg after superkabuki just like the diagram i showed you before.

I also did a test on superkabuki's -t option, though I'm not sure when to use it. 

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ superkabuki -i udp://229.0.0.7:9000 -o IRD_w_time_signal.ts -t

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Reading rcvbuf_size of 212992
Setting rcvbuf_size to 3489660928

Output File:    IRD_w_time_signal.ts
PMT Section Length: 55
Program Number: 671
PCR PID: 6119
Program Info Length: 12

Added Registration Descriptor:
    b'\x05\x04CUEI'

Found Streams:
    Stream Type: 27  PID: 6116  EI Len:  20
    Stream Type: 3  PID: 6117  EI Len:  0

Added Stream:
    Stream Type: 134 PID: 134 EI Len:  0

After stopping the above command I checked the ts file No time signal is inserted, but only the SCTE35 stream is added.

(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive IRD_w_time_signal.ts
(ffmpeg611_python311) imagecuetone@imagecuetone-Virtual-Machine:~/detect_stream_insert_35$ threefive show IRD_w_time_signal.ts

Program: 1
    Service:    
    Provider:   
    Pid:        6115
    Pcr Pid:    6119
    Streams:
                    Pid: 134[0x86]  Type: 0x86 SCTE35 Data
                    Pid: 6116[0x17e4]   Type: 0x1b AVC Video
                    Pid: 6117[0x17e5]   Type: 0x3 MP2 Audio

Did I miss any steps? Or is it necessary to use ffmpeg to copy over the timestamps before putting the stream into superkabuki ?