Map Stream MJPEG instead of series of images

[tonysprenk]

Is your feature request related to a problem? Please describe. I want to add the map stream to HomeKit. This works but only for the preview image. I believe this is because the stream created by this integration is not an actual MJPEG stream but rather just a series of images.

Describe the solution you'd like It would be nice if the stream of the map image was in H264 (HomeKit compatible) or at least actual MJPEG rather than a series of images so that home assistant can decode it properly for HomeKit.

Describe alternatives you've considered I've tried encoding with go2ryc but it does not work as the stream provided by this integration is not actual MJPEG.

Additional context Thank you for the great integration!

[Tasshack]

Can you share the url that you are using to access the map image?

[Tasshack]

Url format should be; {HOMEASSISTANT}/api/camera_stream_proxy/camera.{ENTITY}_map?token={TOKEN}

Not; {HOMEASSISTANT}/api/camera_proxy/camera.{ENTITY}_map?token={TOKEN}

[tonysprenk]

Yes I'm using:

{HOMEASSISTANT}/api/camera_stream_proxy/camera.{ENTITY}_map?token={TOKEN}

But I'm actually injecting a long lived access token authorization header because the token in the url refreshes every so often. Either way the stream I believe is just a series of images.

[Tasshack]

I believe is just a series of images.

You are wrong it is an mjpeg stream. You cannot just change images from browser without injecting an header thus it makes it an mjpeg stream.

[Tasshack]

Try with this. https://www.home-assistant.io/integrations/proxy/

[tonysprenk]

The result is the same. When I put this stream into HomeKit I get an error in home assistant saying:

Camera has no stream source

When I put the camera component from the Dreame integration I get the same error.

[Tasshack]

That means HomeKit does not support mjpeg stream. You can try to use ffmpeg camera proxy to convert it to h264.

[Tasshack]

Looks like you have to convert it yourself using ffmpeg.

https://github.com/home-assistant/core/issues/100768

[tonysprenk]

Yes I saw this as well but when I try to convert with go2rtc I still get no camera stream found.

[tonysprenk]

Whatever I do: convert with ffmpeg, convert with go2rtc, use the built in conversion in the HomeKit integration. I always get a "source has no stream error". Not sure what my next steps should be. Any suggestions would be highly appreciated. I think there would be quite a bit of interest from people using HomeKit. Thanks!

[Tasshack]

MJPEG camera stream implementation is exactly same with the official home assistant camera component.

https://github.com/home-assistant/core/blob/dev/homeassistant/components/camera/__init__.py#L259 https://github.com/Tasshack/dreame-vacuum/blob/dev/custom_components/dreame_vacuum/camera.py#L611

So I have no idea it is not working. Only difference is that the integration generates png images instead of jpegs but ffmpeg should handle that correctly. I can further investigate if you can provide an integration that generates mjpeg stream compatible with HomeKit.

[tonysprenk]

I think the fact that they are png images is the clue. When I do the following command in terminal outside of home assistant it works as expected.

ffmpeg -re -loop 1 -headers "Authorization: Bearer long_lived_access_token" -i "http://homeassistant.local:8123/api/camera_proxy/camera.x40_ultra_complete_map" -vf "fps=1" -vcodec libx264 -r 15 -pix_fmt yuv420p output.mp4

Note the loop argument. FFMPEG needs to be told to keep looping even if the image is not refreshed constantly. Does the Dreame integration only refresh the image when a change is detected? Or does it serve a new image every second for example? The only thing I need to do now is to find a way to implement the above command in home assistant. I haven't been successful yet as I'm getting 400 errors.

[Tasshack]

Does the Dreame integration only refresh the image when a change is detected? Or does it serve a new image every second for example

It buffers a new image only when it is rendered and not periodically.

The only thing I need to do now is to find a way to implement the above command in home assistant.

Try ffmpeg camera platform https://www.home-assistant.io/integrations/camera.ffmpeg/

[Tasshack]

You can also drop the frame rate limit to 1 with -r 1 param.