[FR] Add HLS stream support

geNAZt commented 2 years ago

Is your feature request related to a problem? Please describe. Sort of. On my Raspberry PI 4 when using a mjpeg webcam (which is default) it caps the network port and its nearly unuseable (via SSH for example). Also this setup will never work over the network (like on mobile over the internet).

Describe the solution you'd like I would like a webcam type added which loads the HLS polyfill for the video HTML5 tag ( https://github.com/video-dev/hls.js/ ). This would allow to have restreamer (for example) configured to provide a HLS encoded stream from the webcam (in my case C920) hardware encoder for h264 to HLS.

Describe alternatives you've considered Reducing the JPG quality to the bandwidth limitations of my mobile device which resulted in a blob of squares. Also i reduced the resolution from 1080 to 720p which reduces quality even on my local network.

Additional context

KwadFan commented 2 years ago

Hello geNAZt, first of all, thank you for your Request. We are working currently on a new 'webcamd'. This will have multiple Streamtypes available. mjpeg is an kinda outdatet format but since we are limited in Hardware Resources, we have to do some further test to get the best out of it. If you are interested to take a look in the WIP Repo, but I have to say it is a really early stage there are several things missing where I am working on. If you use a MainsailOS 0.5.0 Image you can take a look at it. Install instructions are in the README.md. Basicly it replaces the now used webcamd. The whole Documentation is missing, unfortunatly you are a bit on your own at this stage. Used Streamservices are mjpg_streamer ( for compatibilty reasons ), ustreamer and v4l2rtspserver. Last named is capable of rtsp, hls, mpeg4-dash. A webrtc service is also planned but not implemented yet. Please take a look at the Repo of the named streamservices.

Here we go: crowsnest v4l2rtspserver ustreamer

Regards

dragonnn commented 2 years ago

Hi! I didn't yet look into HLS.js, I am using currently jmuxer.js with my own streamer: https://gitlab.com/dragonn/h264rcam It is a fresh project so not really polished yet, working on a README. I will see if it could work with HLS.js. My streamer has quite good performance, it can run stable 20FPS 1024x576 on a Pi Zero with only 8% CPU usage when a client is connected, about 2% when no clients are connected.

meteyou commented 2 years ago

do you have more informations to jmuxer.js? pls write a short feedback which lib do you prefer...

dragonnn commented 2 years ago

I found about jmuxer.js here https://github.com/dans98/pi-h264-to-browser it worked quite well but used much more CPU (about ~18%) and sometimes did misfire frames (old frame jumps into feature). jmuxer.js is definitely really easy to use, only a few lines and you have a working stream. HLS.js from a quick look does more support "true live streams", jmuxer.js only needs a simple websocket where the backend does sends basic raw h264 frames with for pi cam is super easy. Only issue I had with jmuxer.js was that a new client connects to an already running stream it will not pick up the stream, they are two possible solutions with I tested and decided on one of them:

restart the stream when a new client connects, this cause the already running stream to blackout for a sec, not a huge issue, huge problem was that at least on a Pi Zero restarting the camera stream more then ~5 times ends with a firmware transaction timeout with locks the program complete out, only reboot can recover from that (but even a reboot doesn't success, needs a power cycle to finish the reboot).
after the above solution wasn't really an option I found out that storing the two first frames when the camera stream starts and then sending them to every new client with connects does work and the new client picks up the stream. A workaround but it works well.

I am not sure what pi-h264-to-browser does for getting that work, didn't find anything in the code, I think it creates multiple streams? Not sure about that.

lixxbox commented 2 years ago

We are working on a way to integrate a webcam stream via WebRTC in our side project "crowsnest". In our tests we found that the latency is much lower than HLS.

Unfortunately, the feature is dependent on Crowsnest, which is why I set the status to "blocked" for now. We will keep you up to date.

leigh-johnson commented 1 year ago

:wave: Hey folks, creator of PrintNanny.ai here - I'd like to contribute to getting HLS working in Mainsail.

For context, PrintNanny runs a WebRTC stack with HLS as a compatibility fallback. I strongly prefer hardware-accelerated h264 encoding to JPEG, since PrintNanny runs computationally-expensive computer vision tasks on the Raspberry Pi - every optimization matters!

I'm happy to propose a PR to get the conversation started. =)

meteyou commented 1 year ago

Hey @leigh-johnson! I sent you a private message via discord to not give false hope too soon in public.

meteyou commented 1 year ago

this FR is fixed with this PR: https://github.com/mainsail-crew/mainsail/pull/1258

mainsail-crew / mainsail

[FR] Add HLS stream support #387