Remove RTSP usage when streaming

[deanljohnson]

Ideally, when processing a streaming video we would take input through stdin and send output through stdout.

Stdout: This is the easy one to fix - I just have to output in the same format that we are converting the RTSP output to. For reference, this is the ffmpeg call to convert RTSP to a browser compatible format that we are using right now "ffmpeg -i rtsp://localhost:554/processed.mp4 -f webm -vcodec vp8 pipe:1".

Stdin: This is the tricky one. When using webm input, both ffprobe and ffmpeg need the header portion of the input file. Unfortunately, reading from stdin removes that information from the pipe. This means that once ffprobe reads the header portion it is removed from the pipe and then ffmpeg fails to pull frames. We don't know exactly how long the header information is going to be so we can't just duplicate the beginning. What I would have to do is create two named pipes, duplicate stdin into each, send one named pipe to ffprobe, one to ffmpeg. I don't see a simple way to implement this is the current code layout right now so it will take a bit of engineering. Hope it will work lol

[deanljohnson]

This stdin stuff is ridiculous.

I was trying to get away with having one pipe for ffprobe and sending stdin to ffmpeg. I'd need to peek at the data on stdin without removing the data from stdin and copy it into a named pipe for ffprobe. Unfortunately - you cannot peek more then one character at a time on stdin even if you muck around with its internal buffer. Can't be done.

Okay - two named pipes. We have to solve duplicate pipe names and such for one, but thats a minor issue. Now - to get data onto these pipes I need 3 threads. One for reading and two for writing because the FIFO buffers could easily become too full and block the writers and since the two reads happen at different times I have to buffer the reads off of stdin and synchronize the writes to the pipes. I don't know how much data to buffer, so I have to handle a dynamically sized buffer. I can't block one writer when the buffer is full because I don't know the byte at which images line up on the input and so arbitrarily blocking and then continuing with the latest pipe info will trash the images that ffmpeg recieves.

[gregoryjjb]

Yikes. It wouldn't be the worst thing to just stick with rtsp if we have to. I think a bigger problem is the huge lag you were mentioning, but I don't know how to really debug that since I can't reproduce.

[deanljohnson]

Just want to document the attempt at using tees to solve this:

Not looking hopeful that tees will fix our problem. I can use them to get stdin sent to two named pipes BUT named pipe writes are blocked until a reader opens the pipe. What this means is that tee outputs to two named pipes, I then startup ffprobe which opens the first named pipe but tee is still blocked because the second named pipe (ffmpeg) hasn't been opened yet. I can't startup ffmpeg (opening the second named pipe) until ffprobe completes. I can't do two instances of tee because tee will consume the input on stdin and so the second one wont have input. Maybe I could use a tee to output to two named pipes, then send the second named pipe to another instance of tee which outputs to another named pipe which is then sent to ffmpeg, but this is getting stupidly complicated.

[gregoryjjb]

Alternate attempt: The server calls ffprobe to get the frame information, then passes that into the cvprocessor so it only needs 1 pipe.

Problem: The ffprobe call still takes some indeterminate amount of time, and if the cvprocessor doesn't get the beginning of the video file it's not happy. I tried taking the stream and putting it through another instance of ffmpeg setting every frame as a keyframe with ffmpeg -i pipe:0 ... -g 1 pipe:1 and then passing stdout of THAT into cvprocessor, but it still didn't work if there was any delay.

The only way I could get the cvprocessor to work was to hard code the frame data in the call, allowing me to skip the ffprobe step; that way cv gets the beginning of the file. I was still running into some encoding issues (the video was very distorted initially, and came out somewhat purple looking) but didn't really look into solutions for that since the ffprobe step would break it anyway.