[Q] fork disturb ip-stream

[neo7530]

I like to do dual-output of my dvb-stream. One stream should do udp-multicast, and the fork of this stream should do output to my dektec-device.

with this both output streams are working for about 5 Minutes, then udp multicast get more and more glitches in video and get unwatchable. the dvb-c output via dektec remains clean.

.........
tsp.exe -v -b 50780000 ^
-I file  ^
-P regulate ^
-P filter --max-payload-size 0 --negate -s ^
-P scrambler hd1 --ecmg 10.24.10.78:8888 --super-cas-id 0x0B020000 --channel-id 0x1010 --pid-ecm 0x1EC1 --cp-duration 10 --access-criteria 01ffffff --stream-id 0x1001 --private-data 4E656F566973696F6E73 ^
-P scrambler hd2 --ecmg 10.24.10.78:8888 --super-cas-id 0x0B020000 --channel-id 0x101F --pid-ecm 0x1EC2 --cp-duration 10 --access-criteria 0A --stream-id 0x1002 --private-data 4E656F566973696F6E73 ^
-P scrambler hd3 --ecmg 10.24.10.78:8888 --super-cas-id 0x0B020000 --channel-id 0x1010 --pid-ecm 0x1EC3 --cp-duration 10 --access-criteria 01ffffff --stream-id 0x1003 --private-data 4E656F566973696F6E73 ^
-P cat -c -a 0x0B02/0x11F2 ^
-P datainject -r -s 10.24.10.75:9999 -b 50000 -p 0x11F2 ^
-P datainject -r -s localhost:10000 -b 1000 -p 0x14 ^
-P svrename -n "NeoVision 1" -p "NeoVision" hd1 ^
-P svrename -n "NeoVision 2" -p "NeoVision" hd2 ^
-P svrename -n "NeoVision 3" -p "NeoVision" hd3 ^
-P svrename -n "NeoRadio 1" -p "NeoVision" radio1 -t 2 ^
-P svrename -n "NeoRadio 2" -p "NeoVision" radio2 -t 2 ^
-P svrename -n "NeoRadio 3" -p "NeoVision" radio3 -t 2 ^
-P fork "tsp -O dektec -d 0 -m 256-QAM -f 666000000 --symbol-rate 6900000" ^
-P filter -n -p 0x1FFF ^
-O ip -l 10.24.10.75 224.10.10.10:10000 -t 5 -e

if i remove the fork part, then ip stream will be stable.

i even tried the other way:

........
-P svrename -n "NeoRadio 3" -p "NeoVision" radio3 -t 2 ^
-P fork "tsp -P filter -n -p 0x1FFF  -O ip -l 10.24.10.75 224.10.10.10:10000 -t 5 -e" ^
-O dektec -d 0 -m 256-QAM -f 666000000 --symbol-rate 6900000

I got the same result then.

Do you have any idea, what's going wrong here?

Kind regards, Marco.

[lelegard]

Hi @neo7530

Try -P fork -b 7 (or even 0)

$ tsp -P fork --help
...
  -b value
  --buffered-packets value
      Specifies the number of TS packets to buffer before sending them through
      the pipe to the forked process. When set to zero, the packets are not
      buffered and sent one by one. The default is 500 packets in real-time
      mode and 1000 packets in offline mode.

The default value maximizes the throughput at the expense of latency. In the case of IP output, all chunks of packets shall be passed asap.

[neo7530]

Thank you. The lowest value i can go is -b 1. It works better, but not perfect yet. Will try this with linux.

Kind regards Marco.

[lelegard]

At 50 Mb/s (your rate), you send 33224 packets per second.

Tsp was optimized to keep everything in one single multi-threaded process with one single common large packet buffer. Even when you have an intense activity, you switch between threads but you keep the same virtual memory space.

When you use fork, you create another process. So, in addition to switching between threads, you also switch between processes and, consequently, virtual memory spaces. And when you want to improve the latency with one packet at a time (-b 1), you do that 33224 times per second.

If you use larger pipe buffering sizes, you reduce the process scheduling and memory management pressure. But you increase the packet latency (as you experienced). You can't have everything...

Windows is known for its heavy process management, which makes things worse. Linux (and all Unix systems) have a more lightweight process management. So, switching to Linux can only improve the situation.

That being said, TSDuck is a very generic lab toolkit. It was designed to be highly flexible for all kinds of experimental environments. This means that it is tuned for flexibility over efficiency. Real-time streamers for high bitrates have a different, optimized, dedicated architecture. They are less flexible than TSDuck, they can do only one thing, but they do it extremely well, better than TSDuck.

This is why TSDuck may not be the right tool for high bitrate streaming in production. You can't have everything for free...

[neo7530]

That being said, TSDuck is a very generic lab toolkit

But a very good toolkit. Just to say... 👍

Thank you for your work.