esp32 cam rtsp video stream

[dabos1]

Hello It would be very nice. If the esp32 cam could stream video over rtsp.

Thank

Have you looked for this feature in other issues and in the docs?
yes Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. i cannot stream videos over rtsp Describe the solution you'd like
A clear and concise description of what you want to happen. That the esp32 cam streams videos over rtsp Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

(Please, remember to close the issue when the problem has been addressed)

[ascillato]

By now, if you add your ESP32CAM as a mjpeg camera in your DVR (as http://IP:81/stream) it works. So, it isn't really needed to add another protocol.

The only thing by now is that you need to open Tasmota web page, at least once, so that the stream will start. I'm going to make a PR adding an option for not needing that.

Hope this helps.

[teixeluis]

Hi @ascillato, in my ESP32-CAM AiThinker board running Tasmota 8.5.01, I also get the behavior of having to refresh the page once again, so that the stream starts. The following sequence of actions in particular seems to also reproduce the issue:

1. open the main page;
2. stream may or may not appear;
3. Go to the console;
4. disable the stream (wcstream 0)
5. turn the stream back on (wcstream 1)
6. open the main page again;
7. stream won't appear (broken image link);
8. refresh the page
9. stream will appear.

Sometimes with this set of steps, at the end the stream never starts, until I cycle the command again (wcstream 0; wcstream 1).

Being possible to control the stream independently from accessing the management UI would be a significant improvement.

For power saving, the streaming loop could be automatically shutdown after an absence of http requests (and turned on as soon as these start hitting the stream URL). Would this make sense? (it is still a 50 mA difference and less heat buildup).

Cheers

[ascillato]

Being possible to control the stream independently from accessing the management UI would be a significant improvement.

Totally agree. I'm already using this as a security camera at home and the esp32cam is very reliable. No hungs nor freezes, very smooth image. Very happy with this and the work done by @gemu2015 . At this moment I'm moving to another home. After that I want to make a PR for independent stream.

[teixeluis]

Yes, its a nice little package. And the hardware is quite cheap. I think Tasmota can have some edge in supporting this as well. Awesome.

[gemu2015]

by the way you may directly open a stream without ever calling the web Gui

by e.g. http://192.168.178.151:81/stream

rtsp would be some extra code

[gemu2015]

sorry i forgot to say that this is only true with a script initializing the cam

[gemu2015]

here is a sample script that also has a motion and brightness detector

>D IP=192.168.178.151 res=0 w=0 h=0 mot=0 bri=0 >B ; init cam with QVGA res=wc(0 4) ; get pixel size w=wc(3) h=wc(4) ; start motion detector, picture every 1000 ms mot=wc(6 1000) >S if wific>0 then ; when wifi up, start stream res=wc(5 1) endif ; get motion detect diff value mot=wc(6 -1) ; get picture brightnes bri=wc(6 -2) >W

motion diff = %mot%
brightness = %bri%
; show stream on WEBUI $\
$\ $\
\webcam stream

[teixeluis]

hi @gemu2015 thanks for the feedback. Via a script are we alson able to instead of keeping the camera active for performing motion detection and such, to only activate the camera when the request to obtain the stream is received? And when the rtsp teardown happens, shutdown the streaming loop (equivalent to wcstream 0)?

Thank you

Cheers

[gemu2015]

@teixeluis you can enable and disable the stream on demand, also the motion detector etc.

currently i do not plan an rtsp streaming mode i am busy with other projects, maybe @ascillato has this on his list.

[ascillato]

I don't have in my list adding RTSP as the actual MJPEG stream is more than enough and working really good. Great job @gemu2015 !!! What I want to add is to be able to initiate the stream using also rules, so in the case you want to use an esp32cam as a security camera (and use the precompiled version of Tasmota), you can enable the stream at boot time.

[teixeluis]

I have incorrectly mentioned rtsp (that is the feature request of this ticket). In this case, the idea would be when the client closes the socket (e.g. when the user navigates away from the site or closes the browser), we would not need to keep the camera resources running. I can observe that after the client application is closed (i.e. browser), there is still extra power being consumed by the ESP32. It only drops when I manually execute the wcstream 0 command. It is a drop from about 200mA to 150mA if we had it initially set to wcresolution 10.

Cheers

[ascillato]

@teixeluis

the idea would be when the client closes the socket, we would not need to keep the camera resources running.

Great idea. May be we can add a rule trigger (and also in scripting) to know when the webUI is not being used and if the user wants, a rule to turn off the stream, can be added (very useful to save energy if the esp32cam is not used as a security camera).

[teixeluis]

very useful to save energy if the esp32cam is not used as a security camera

yes, even then, for the cases where it is difficult to provide good ventilation to the chip, not generating more heat than necessary can make a difference.

[teixeluis]

e.g. in a security camera that is supposed to be triggered by an external motion detector.

[KaiGrassnick]

I would also like to see RTSP as a feature which would allow to include the ESP32CAM in HASSIO and show it on an Alexa device.

You can only stream that using RTSP but not mjped etc. So there would be a real use case, wich at the moment is just not possible.

see: https://www.home-assistant.io/integrations/alexa.smart_home/#camera and the information about the stream component: https://www.home-assistant.io/integrations/stream/

[gemu2015]

ok i implemented RTSP (initial support using port 8854) please try my fork: https://github.com/gemu2015/Sonoff-Tasmota after some people have tested i will do a pr

define ENABLE_RTSPSERVER

before start you must setup alle params by wc commands and enable stream

new cmd wcinit may be used by rules to enable wc without webgui

[teixeluis]

@gemu2015 awesome. Looking forward to test it. Can we still stop the stream via wcstream 0 (or via different method)?

Thank you

Cheers

[gemu2015]

rtsp can stop the rtsp stream via rtsp cmd , wcstream 0 stops the HTTP stream

[gemu2015]

addendum

i tested with VLC you must enter the url: rtsp://192.168.178.162:8554/mjpeg/1 (replace ip with yours)

[teixeluis]

Hi @gemu2015,

I have built from your fork, and added:

#define ENABLE_RTSPSERVER

in tasmota/user_config_override.h

Set:

tasmota32-webcam

in platformio_override.ini

And when I call the rtsp command, I get:

18:17:16 CMD: Group 0, Index 1, Command "RTSP", Data ""
18:17:16 RSL: RESULT = {"Command":"Unknown"}

It appears that the rtsp module is being built into the firmware, as I get the following:

18:08:42 NTP: UTC 2020-10-31T17:08:41, DST 2020-03-29T02:00:00, STD 2020-10-25T03:00:00
18:08:44 QPC: Reset
18:08:45 HTP: Main Menu
18:08:45 CAM: Stream init
18:08:45 CAM: Default template
18:08:45 CAM: PSRAM found
18:08:46 CAM: Initialized
18:08:46 CAM: RTSP init

On the other hand it appears that your wcinit command works:

18:21:46 CMD: wcinit
18:21:46 SRC: WebConsole from 192.168.1.80
18:21:46 CMD: Group 0, Index 1, Command "WCINIT", Data ""
18:21:46 CAM: Default template
18:21:46 CAM: PSRAM found
18:21:47 CAM: Initialized
18:21:47 RSL: RESULT = {"WCInit":"Done"}
18:21:47 CAM: RTSP init

Can there be anything missing?

I've also tried opening the stream on vlc, for both:

rtsp://192.168.1.56:8554/mjpeg/1

and

rtsp://192.168.1.56:8854/mjpeg/1

(you made reference to both ports)

no stream appears, and also nothing is logged in the console.

Thank you

Cheers

[teixeluis]

Hi @gemu2015 , well, ended up having success just by doing wcinit prior to connecting VLC to the 8554 port. Just as I have previously noticed, the proximity of the wifi antenna to the camera and/or esp32 board seems to cause problems with video streaming in general, causing the device to become very slow. I've added a pigtail with a further away antenna, and was able to verify it working.

It seems to be a bit unstable when switching resolutions (with the wcresolution command). The stream sometimes does not start again without a restart of the device.

Cheers

[sfromis]

When it comes to stability, my experience is that you need to make sure to have sufficient cooling, as the heavy load can make the ESP32 quite hot. Using my thermal camera, I've seen all of the ESP32-CAM board at around 50 C, when I got it stable by ensuring free airflow on both sides. Must be pretty hot inside the can.

[teixeluis]

@sfromis yes that I a concern too for sure. But I think I haven't crossed that bridge with my project quite yet. Instability starts (low framerate and lack of responsiveness from the web UI) when I approach the Wifi camera from the board, even if the device is still cool. As soon as I move the antenna away from the board it gets back to normal.

In my setup I have adapted a regular doorbell switch to have the camera and esp32 in it:

For now I'm just testing it in free space, so ventilation should not be a problem. But to have it where it is meant to be there will be at least the ventilation and wifi antenna proximity as two challenges here.

[teixeluis]

@gemu2015 does the camera have a still image url?

[sfromis]

.../snapshot.jpg works for me to give a still image, on port 80. .../wc.jpg works too, and .../wc.mjpeg

[teixeluis]

@sfromis confirmed. Works on my end too.

[gemu2015]

just for info: rtsp was already merged into Tasmota recently. No need to use my fork.

according the slow camera problem. this is mainly related to the camera clock line which is extremely sensitive to EMI and capacitive loads.

unfortunately they choose GPIO 0 for the clock which you will need for flashing.

it is very important to remove any wire from GPIO0 after flashing. Even a 2 cm wire will cause problems.

[teixeluis]

Thanks for the additional feedback @gemu2015. Didn't check to see if it was already merged.

Rgarding the camera, from experimentation in my setup the stabilty appears to be proportional to the distance between the wifi antenna and the board. I've moved the antenna as far away as it is feasible within the enclosure, and I am already getting better results even though the stream gets interrupted once in a while:

Tried covering camera flatflex cable with aluminum, but it seemed to make the it worse (perhaps it increases the capacitance), so removed it afterwards:

I tried integrating with Home Assistant (Hass.io) using the Camera integration, and apparently it is not handling the format of the stream (perhaps it does not support mjpeg (?)):

Stream connection failed: rtsp://192.168.1.56:8554/mjpeg/1
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/stream/worker.py", line 48, in stream_worker
    _stream_worker_internal(hass, stream, quit_event)
  File "/usr/src/homeassistant/homeassistant/components/stream/worker.py", line 65, in _stream_worker_internal
    container = av.open(stream.source, options=stream.options)
  File "av/container/core.pyx", line 354, in av.container.core.open
  File "av/container/core.pyx", line 225, in av.container.core.Container.__cinit__
  File "av/container/core.pyx", line 257, in av.container.core.Container.err_check
  File "av/error.pyx", line 336, in av.error.err_check
av.error.InvalidDataError: [Errno 1094995529] Invalid data found when processing input: 'rtsp://192.168.1.56:8554/mjpeg/1'

The still images work great though. Will give a try to the mjpeg integration to see if it works ok.

Thanks

[gemu2015]

video about GPIO 0 https://www.youtube.com/watch?v=NvmyCBbTGPs&feature=youtu.be

i only got to work rtsp with VLC. It seems to be a not so common format.

i tried several IOS camviewer apps, no success

the code is based on this project : https://github.com/geeksville/Micro-RTSP

[teixeluis]

@gemu2015 thanks for the references. Indeed this inspired me to do the following small change, wich ended up doing a big difference. Simply by cutting away part of the connector that is on the perfboard and leaving the gpio0 pin standing in the air (even though previously it was not connected to anything) I am getting much better results without having the antenna away from the doorbell switch:

Perhaps the RF field just exacerbates the interference on an already very sensitive clock pin.

Thanks

[sfromis]

I'm on the verge of snipping off the IO0 pin completely, no need for more than handheld grounding there.

[teixeluis]

@sfromis indeed. Thought about the same. Will do it also, if this proves to be still insufficient. Pretty bad board design to select this pin for this purpose. Such high speed clock line should never be routed to nothing else than between the MCU and the camera.

[kugelkopf123]

@gemu2015 How about to terminate io0 with a 100k resistor to GND or 3,3v? Ever tried something like this?

[teixeluis]

hi @kugelkopf123 I tried to do that (a 10K resistor pulling down to GND and in a second test ,to VCC) and didn't see any difference..

[gemu2015]

that was the first thing i tried with various resistor values.

However this error doesn't affect all cams. i have 3 of this type now and one (bought separately) does not show this error. even if i touch GPIO 0 with my finger this one is working completely unaffected this unaffected board looks exactly like the other 2 but the camera module is different 8225N-V2.0 171025 => no error TY-OV2 640-V2.0 => shows the error

[teixeluis]

Different cameras may have slightly different pixel clock input impedances, causing different behaviours.. In my setup I have also trimmed the IO0 pin just in case, and it is working great so far.

[gemu2015]

the input capacity of the cam modules seems to be different. while the perfect working cam shows a square wave on the clock line (20 MHz) the bad working shows a sinusoidal waveform which distorts severely when touching with the finger

one of my bad cam modules now completely stopped working, probably bad production quality ?

[teixeluis]

In 4 OV2640 cameras I have tested, 2 don't work and one seems dodgy (sometimes is not recognized). I have more which I haven't tested yet. Bottom line: better order more cameras than you need, to compensate for the defect rate.

[ascillato2]

Closing as this feature has been added by @gemu2015 ( #9575 )

Thanks!!! great work.