search

131 / h264-live-player

A live h264 player for the browser (ideal for raspberrypi / raspicam )
MIT License
1.07k stars 250 forks source link

Communication Protocol between Server and Client #35

Open smanschi opened 7 years ago

smanschi commented 7 years ago

Nice project! I want to use my own WebSocket Server to send the H264 frames to the client. Hence, I want to use your live-player only for displaying the frames in the browser. When I click on "Start Video" in your index.html sample I receive a REQUESTSTREAM WebSocket message on the server side and start sending the frames, but nothing is displayed in the browser. I send a baseline encoded h264 stream with a normal bitrate (I get the frames from an RTSP camera). The messages are send as a bytestream (opcode 130). Do I have to consider something else? I also tested an mp4 muxer (muxing the h264 frames on the client side to a mp4 file which is then displayed). This works, so I assume the frames are send correctly..

131 commented 7 years ago

resolution ? frame type ? frame headers, is it possible for you to provide a an archive with raw frames (as in raw00001.h264,raw00002.h264,raw00002.h264 & a frame rate so 'ill try this by my side ?

Also, i'm curious with the working client side mp4 muxer you used, could you point me the project you used/ how you achived it ?

smanschi commented 7 years ago

Resolution is 640x480@25fps. I-Frame interval is 50. Here is a link to a sequence of send frames: Link. I started debugging the liveplayer in firefox. It seems like the NAL types are detected properly. It detects PPS, SPS, I-Frame and calls the avc.decode() function. That's all I figured out so far.

The h264 muxer I found on Github. The muxing to mp4 seems to work fine, but the player itself is rather basic (e.g., if the buffer is empty the video restarts etc..)

131 commented 7 years ago

I'll be doing tests in https://github.com/131/h264-live-player/blob/frames/server-frames.js. Feel free to give hand !

smanschi commented 7 years ago

I'll be on holiday for the next three weeks. In April, I will continue working on my project and will try to investigate this issue further.

adrhc commented 6 years ago

hi, how to use server-frames.js? Please, some short help here, thank you.

qmfrederik commented 6 years ago

Hello,

I gave the server-frames.js sample a quick try, but it doesn't seem to work. I checked out the frames branch and the H264 frames in the frames/ subfolder.

Then, I ran node server-frames.js and connected to http://locahost:8080 using my browser. When I connect and hit Start Video in the browser, I get console output (see below) but no video is rendered - just an empty screen.

I can share (some of) the frames (or at least what I believe to be H264 frames) if you want.

Meanwhile, is there anything I can do to troubleshoot what went wrong? I don't see any output in the JavaScript debugger console, unfortunately.

qmfrederik commented 6 years ago

Just to be sure - based on:

https://github.com/131/h264-live-player/blob/c5bfaaf0f8cc38c35f6030b3b7377f8b12e2cc15/lib/_server.js#L51

I'm assuming the frames are really NAL units, that they should start with the NAL separater (0x00000001), and that the data in the NAL unit should use emulation prevention.

I also updated the width & height here:

https://github.com/131/h264-live-player/blob/frames/lib/frames.js#L55

Unfortunately to no avail.

qmfrederik commented 6 years ago

Update - in my case, the video is encoded using the High profile and I realized that the underlying decoder does not support the High profile.