Suggestion: Subprocess PIPE input

lars18th commented 5 years ago

Hi @Barracuda09 ,

Instead of the most complex request at #63, I ask for another enhancement that can provide a great of flexibility:

In addition to the FILE input (?msys=file&uri=test.ts), it will be interesting to support CHILD inputs. Example: ?msys=child&exec=ffmpeg&params="... -f mpegts pipe:1". The idea is conceptually very simple: the SATPI server starts the process exec with the parameters params and reads the stream from the stdout PIPE of the child process.

What you think? Regards.

lars18th commented 5 years ago

Hi @Barracuda09 ,

I'm trying to modify the current file input to read from a pipe. The idea is to call to popen(filePath, "r"); with the name of the subprocess to execute. The problem is that the popen() function uses a FILE class, instead of a filestream. So I can't find a solution.

Please, can you enhance your FILE input to support a PIPE subprocess? I feel this will be a very useful functionality to support external transcoding in realtime. You agree? Regards.

Barracuda09 commented 5 years ago

Yes it would an nice enhancement. I will try to look at this.

Barracuda09 commented 5 years ago

Hi @lars18th

I have done a proof of concept for childpipe reader with hard coded 'cat test.ts' as input.

The syntax will be something: ?msys=childpipe&exec=cat%20test.ts

Barracuda09 commented 5 years ago

Hi @lars18th

I have now included to read exec string from input url.

The syntax is for example cat test.ts => ?msys=childpipe&exec=cat%20test.ts

lars18th commented 5 years ago

Hi @Barracuda09 ,

Thank you! It's very promissing... almost now &exec=cat%20test.ts works.

However, some suggestions:

Please, add some start option to enable (and put it by default disabled) this new msys=childpipe mode. It can be a security hole. And I feel most users will prefer to have it disabled. You agree?
Please, decode the exec parameter. At time this doesn't work ?msys=childpipe&exec=cat%20%3Fdata%3Ftest.ts or ?msys=childpipe&exec=cat%20/data/test.ts. We need to support special characters like / for example.

I'll do more tests and I'll post results here. Regards.

Barracuda09 commented 5 years ago

Hi @lars18th

Yes both options will be implemented

lars18th commented 5 years ago

Hi @Barracuda09 ,

Thank you for the improvement. However, in my case it doesn't work! :cry:

Here the mapping.m3u:

#EXTINF:-1 satip-freq="10744", Translation to: ChildPIPE exec=wget -qO- http://127.0.0.1:8001/test
rtsp://127.0.0.1:8554/?msys=childpipe&exec=wget%20-qO-%20http%3A%2F%2F127.0.0.1%3A8001%2Ftest

And here the LOG:

[                        src/StreamManager.cpp:170] Found StreamID x - SessionID: 0243950177
[                               src/Stream.cpp:215] Stream: 0, StreamClient[0] with SessionID 0243950177 for dvbs
[                   src/input/dvb/Frontend.cpp:422] Stream: 0, Parsing transport parameters...
[                 src/input/Transformation.cpp:150] Stream: 0, Request Transformed
[                 src/input/Transformation.cpp:183] Stream: 0, Request Transformed to SETUP rtsp://127.0.0.1:8554/?msys=childpipe&exec=wget%20-qO-%20http%3A%2F%2F127.0.0.1%3A8001%2Ftest RTSP/1.0
[               src/input/dvb/FrontendData.cpp:255] Stream: 0, Not supported delivery system
[                   src/input/dvb/Frontend.cpp:429] Stream: 0, Parsing transport parameters (Finished)
[                               src/Stream.cpp:270] Stream: 0, Found Streaming type: RTSP Unicast
[                   src/input/dvb/Frontend.cpp:433] Stream: 0, Updating frontend...
[                   src/input/dvb/Frontend.cpp:720] Stream: 0, Opened /dev/dvb/adapter0/frontend0 fd: 10
[                     src/socket/TcpSocket.cpp:109] HTTP Client 192.168.210.101:41689 Connection closed with fd: 8
[                          src/HttpcServer.cpp:204] RTSP/1.0 200 OK

And note the Not supported delivery system. Why it doesn't work? I started the SATPI with ./satpi --no-ssdp --childpipe --rtsp-port 8554 --no-daemon

Barracuda09 commented 5 years ago

Frontend 0, or any other, is 'ChildPIPE TS-Reader'

lars18th commented 5 years ago

Frontend 0, or any other, is 'ChildPIPE TS-Reader'

Hi,

I need to do more tests, but for the first time I has started to work! :hugs:

Now I "disabled" (in the web interface) the frontend 0 (the real tuner), and in the frontend 3 (Child PIPE) I enabled "Advertise as" (DVB-S2) and "Transformation enable". I suspect, that these options are required (so I suggest to comment about them in the mapping.m3u file).

Regards.

Barracuda09 commented 5 years ago

Yes that is all required. I will do this.

lars18th commented 5 years ago

Hi,

I confirm it:

To use the ChildPIPE with a transformation, the real frontend 0 requires to have the Tranformation enable option DISABLED. And the ChildPIPE frontend 3 requires to have it ENABLED.

So, the problem seems to appear when two different frontends have this option enabled. In that case the SATPI process selects the first one.

But now I see that the Transformation Mapping M3U option is per frontend. I can use then two different files to overcome this problem?

Barracuda09 commented 5 years ago

Well first of all it will check the real frontend so if you request msys=dvbs2 and it finds this it will always use this one. To transform it will check the freq requested if it is in the mapping.m3u file.

Barracuda09 commented 5 years ago

Maybe if I put the 'fake' frontends first in the list. Then that would fix this (maybe)

lars18th commented 5 years ago

Maybe if I put the 'fake' frontends first in the list. Then that would fix this (maybe)

Maybe... however, I feel it's preferable to have them first, as then the frontend number will not depend in this case of the number of the real tuners. So fe=0-2 will be the same every time.

Regards.

Barracuda09 commented 5 years ago

Yes, I forgot you can always request fe=x to have it point to the correct one.

lars18th commented 5 years ago

Yes, I forgot you can always request fe=x to have it point to the correct one.

True. But the use of the fe= parameter is not recommended in the specs for regular clients. So I recommend to overcome the issue whitout using it.

lars18th commented 5 years ago

Hi @Barracuda09 ,

Another issue detected with the ChildPIPE frontend: The RTCP packets are not confirmant. See the content (from the LOG) when using it with the real frontend disabled and the "transformation" and "advertise" enabled for the virtual frontend:

v=0
o=- 0253037672 0253037672 IN IP4 192.168.1.27
s=SatIPServer:1 0,0,0
t=0 0
m=video 0 RTP/AVP 33
c=IN IP4 0.0.0.0
a=control:stream=3
a=fmtp:33 ver=1.5;tuner=4,240,1,15;exec=None
a=sendonly

So, when using the TRANSFORMATION functionality the a=fmtp:33 ... line requires to be conformant with the selected ADVERSITE mode. If not then some clients refuse to play.

Regards.

lars18th commented 5 years ago

Hi @Barracuda09 ,

I found a SERIOUS bug with the childPIPE !!!

The child process is not killed. When using something like exec=wget -qO- ... then multiple "wget" processes are created (check it with ps -aex | grep wget). In fact, for every new call a new process is created.

Please, check to kill the subprocess every time you close the stream. Thank tou!

Barracuda09 commented 5 years ago

Ok, it should be. I will have a look at this

lars18th commented 5 years ago

Hi @Barracuda09 ,

I close this, as the solution is done in #77.

Barracuda09 / SATPI

Suggestion: Subprocess PIPE input #69