Using GStreamer with h264-live-player on Raspberry Pi

[aplocher]

Hello, has anyone been able to get GStreamer to work with this? I would like to request an official example of how to do it, if possible.

Here's what I've tried:

1. Modified lib/raspivid.js This pipeline should send an h264 camera stream (the raspicam) to STDOUT with a clock overlay added:

   gst-launch-1.0 rpicamsrc bitrate=950000 ! video/x-h264,width=800,height=480,framerate=10/1 ! \
   h264parse ! decodebin ! clockoverlay shaded-background=true draw-shadow=true font-desc="Nimbus Mono" ! \
   omxh264enc ! video/x-h264,width=800,height=480,framerate=10/1,profile=high ! h264parse ! \
   queue ! fdsink

   This works as expected from the command line. However when modifying raspivid.js, it does not seem to work. It does activate the camera (the red LED comes on), and if I change the fdsink to a tcpserversink I can connect to it fine, so gst-launch-1.0 IS working, it's just h264-live-player isn't picking up the data from STDOUT (or isn't recognizing it):

   get_feed() {
       //var msk = "raspivid -t 0 -o - -w %d -h %d -fps %d";
       //var cmd = util.format(msk, this.options.width, this.options.height, this.options.fps);
       var msk = "gst-launch-1.0 rpicamsrc bitrate=950000 ! video/x-h264,width=800,height=480,framerate=10/1 ! h264parse ! decodebin ! clockoverlay shaded-background=true draw-shadow=true font-desc=\"Nimbus Mono\" ! omxh264enc ! video/x-h264,width=800,height=480,framerate=10/1,profile=high ! h264parse ! queue ! fdsink";
       var cmd = msk;
   
       console.log(cmd);
   
       //    var streamer = spawn('/opt/gst/bin/gst-launch-1.0', ['', '0', '-o', '-', '-w', this.options.width, '-h', this.options.height, '-fps', this.options.fps, '-pf', 'baseline']);
       var streamer = spawn('/opt/gst/bin/gst-launch-1.0', ['rpicamsrc', 'bitrate=950000', '!', 'video/x-h264,width=800,height=480,framerate=10/1', '!', 'h264parse', '!', 'decodebin', '!', 'clockoverlay', 'shaded-background=true', 'draw-shadow=true', 'font-desc="Nimbus Mono"', '!', 'omxh264enc', '!', 'video/x-h264,width=800,height=480,framerate=10/1,profile=high', '!', 'h264parse', '!', 'queue', '!', 'fdsink']);
       streamer.on("exit", function(code){
         console.log("Failure", code);
       });
   
       return streamer.stdout;
     }

2. Using the server-tcp.js example

   I have an almost identical gst pipeline, only this time it outputs the h264 stream to any clients connected to a tcp port:

   gst-launch-1.0 rpicamsrc ! video/x-h264,width=800,height=480,framerate=10/1 ! h264parse ! \
   decodebin ! omxh264enc ! tcpserversink port=5001 host=0.0.0.0

   If I connect to this TCP stream with VLC or another instance of GStreamer (locally or remotely) it works well, however I also don't get video with h264-live-player when I try with that.

In example 1, it's using the STDOUT file-descriptor to display an h264 stream, and in the second it's using a TCP h264 stream. I feel like this should be working just fine, but it's not. I was able to get both examples working just fine with raspivid (as your standard code includes), but I need to be able to use GStreamer to add overlays and custom processing plugins.

Is there any apparent reason these commands aren't working with h264-live-player?

Thanks!

[aplocher]

Btw, if I pipe either the raspivid or the gst-launch-1.0 (using the fdsink, example #1) commands to a .h264 file and then run ffprobe on it, this is waht I get:

pi@TG-BBCAM-01:~/src/h264-live-player $ ffprobe -hide_banner gst.h264; ffprobe -hide_banner raspivid.h264
Input #0, h264, from 'gst.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 800x480 [SAR 1:1 DAR 5:3], 25 fps, 25 tbr, 1200k tbn, 50 tbc
Input #0, h264, from 'raspivid.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 800x480, 25 fps, 25 tbr, 1200k tbn, 50 tbc

The only noticeable difference is that SAR / DAR thing that Gstreamer's version has...

[aplocher]

Solved the issue (finally) with this code. The main thing seems to be using Profile=baseline. High and Main would not work for me. Also, not sure if the '-q' is important, but when I dumped the fdsink output to a file, I noticed it included the text at the top until I added it:

"Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock"

"use strict";

const util      = require('util');
const spawn     = require('child_process').spawn;
const merge     = require('mout/object/merge');

const Server    = require('./_server');

class RpiServer extends Server {

        constructor(server, opts) {
                super(server, merge({
                        fps : 10,
                        width: 800,
                        height:480,
                }, opts));
        }

        get_feed() {
                var bin = '/opt/gst/bin/gst-launch-1.0';
                var args = ['-q',
                        'rpicamsrc', 'bitrate=950000', '!', 
                        'video/x-h264,width=800,height=480,framerate=10/1', '!',
                        'h264parse', '!', 'decodebin', '!', 
                        'clockoverlay', 'shaded-background=true', 'draw-shadow=true', 'font-desc="Nimbus Mono"', '!',
                        'omxh264enc', '!', 
                        'video/x-h264,width=800,height=480,framerate=10/1,profile=baseline', '!',
                        'fdsink'
                ];

                console.log(bin +' '+ args.join(' '));

                var streamer = spawn(bin, args);
                streamer.on("exit", function(code){
                        console.log("Failure", code);
                });

                return streamer.stdout;
        }

};

module.exports = RpiServer;

[MadMax411]

            var args = ['-q',
                    'rpicamsrc', 'bitrate=950000', '!', 
                    'video/x-h264,width=800,height=480,framerate=10/1', '!',
                    'h264parse', '!', 'decodebin', '!', 
                    'clockoverlay', 'shaded-background=true', 'draw-shadow=true', 'font-desc="Nimbus Mono"', '!',
                    'omxh264enc', '!', 
                    'video/x-h264,width=800,height=480,framerate=10/1,profile=baseline', '!',
                    'fdsink'
            ];

Hello Adam,

do you have an example for gstreamer options that uses the "videotestsrc" as source?

My try:

    var args = [
      "-q",
      "videotestsrc", "is-live=true",
      "!",
      "x264enc", 
      "!",
      "video/x-h264,width=640,height=480,framerate=15/1,profile=baseline",
      "!",
      "fdsink",
    ];

If I pipe the stream into a file (gst-launch-1.0 ... > dump.h264) and use "ffprobe" to validate the file content I get the following output:

$ ffprobe -hide_banner dump.h264 
[h264 @ 0x55de4370b120] Format h264 detected only with low score of 1, misdetection possible!
[h264 @ 0x55de4370c940] missing picture in access unit with size 39724171
[AVBSFContext @ 0x55de4370bd60] No start code is found.
dump.h264: Invalid data found when processing input

And no stream is playing with these options. The static video file example works fine.

Many thanks in advance!

Markus

[MadMax411]

I found a solution for me:

var args = [
      '-q',
      'videotestsrc', 'is-live=true',
      '!',
      'x264enc', 'speed-preset=superfast', 'key-int-max=6',
      '!',
      'video/x-h264,width=640,height=480,framerate=30/1,profile=baseline',
      '!',
      'h264parse',
      '!',
      'video/x-h264,stream-format="byte-stream"',
      '!',
      'fdsink',
    ];