Allow pushing stream data into JSMpeg

[mandulaj]

I am trying to work this library into one of my project and at the moment, I have once singular issue making it coexist with my current setup. I am already using websockets for sending data back and fourth between the client and server.

My setup: The messages are in JSON and contain a channel field which I use to route them in the receive-event-listener to their corresponding sections of the application - chat, telemetry, control data etc...

I was thinking of creating a new channel for the video data, which I would push then push into the JSMpeg library somehow in order to display it.

However I can not find any way to achieve this. I can only provide a URL to my server which would however also push all the other messages to the JSMpeg.

Basically I am looking into some way to support channels, or pubsub broker. I have a my custom one, but there are multiple approaches out there: https://socket.io/ https://github.com/O3Labs/neo-ws-pub-sub... In order to use any of these methods, there has to be some way to 'push' data into the JSMpeg library.

I envision something along the lines of a player.pushData(data); method which can be called from anywhere like this:

socket.on("message",  (data) => {
    if(data.type == 'video') {
         player.pushData(data.videoData);
    } else if (data.type == 'chat') {
        // Handle chat data.....
    }
   /// ....etc
});

Or when using something like socket.io:

socket.on('video', (data)=>{
    player.pushData(data);
});

// Or even simpler
socket.on('video', player.pushData);

I have looked at the #194 issue, and it looks like it might be related. However still not resolved.

I hope you can provide some advise and help. Thanks

[phoboslab]

You can create a new Source class and provide it to the player. Have a look at the WebSocketSource as a guide. It's pretty minimal: https://github.com/phoboslab/jsmpeg/blob/master/src/websocket.js

I.e. something like this should work (untested):

var SocketSource = function(url, options) {
    this.streaming = true;
    this.established = false;
    this.completed = false;
    this.progress = 0;
    options.socket.on('video', this.onMessage.bind(this));
};

SocketSource.prototype.connect = function(destination) {
    this.destination = destination;
};

SocketSource.prototype.onMessage = function(data) {
    // Pass the data to the destination - i.e. JSMpeg's demuxer.
    this.destination.write(data);
};

// Create the player instance with your SocketSource class. The options
// object gets passed through to your constructor
var player = new JSMpeg.Player("url-ignored", {
    source: SocketSource,
    socket: yourSocketInstance
});

[mandulaj]

Hi @phoboslab ! Thanks, that's looks very promising! I am not that familiar with the WebSocket API. I will try to adapt your solution into my project! If it works, you are a genius 😂

[cyberquarks]

@phoboslab

How about for this setup:

+---------------------+          ____    ____    ____    ____          +----------+
|  Encoder            |  ----\  | TS |  | TS |  | TS |  | TS |  ----\  |  jsmpeg  |
|  emits 1 sec video  |  ----/  |____|  |____|  |____|  |____|  ----/  |  decoder |
+---------------------+           1s      2s      3s   ... n  (time)   +----------+
                                     (websocket transmission)

So not really MPEG-TS streaming as per spec but emitting 1 sec MPEG-1 encoded audio+video and transmitting over websocket how would jsmpeg handle such?

[mandulaj]

Hey @phoboslab I have adapted the method you suggested and it works. The only thing I was stuck on, I had to set this.progress = 1; in order for the image to show.

I am however still having a few new minor issues. For example strangely I have to force disable webGL with disableGl: true. When I use the view-stream.html example, it works with webGL enabled... I think it might be related to #174. Nevertheless, I will have webGL disabled for now. Thank you again. I have received a solution to my problem so I am going to close this issue.