autorope / donkeycar

Open source hardware and software platform to build a small scale self driving car.
http://www.donkeycar.com
MIT License
3.13k stars 1.29k forks source link

Prevent drive Web UI from sleeping #928

Open Ezward opened 3 years ago

Ezward commented 3 years ago

We want to use the web ui on a phone in conjunction with an RC controller. In this use case, the web UI may stay open for long periods without the user interacting with it. On a mobile device like a phone this may result in the phone going to sleep because of lack of user interaction. We want to prevent the phone from sleeping while our web ui is showing in the browser.

Ezward commented 3 years ago

This is the wakelock api in Chrome; https://developer.chrome.com/blog/new-in-chrome-79/#wake-lock

Ezward commented 3 years ago

In terms of video, the web server code serves a multipart image type to the /video endpoint and writes jpegs to the connection; https://github.com/autorope/donkeycar/blob/b1af0aacf488c938231912f89c4bfa5c2b11c649/donkeycar/parts/web_controller/web.py#L284

The web client code connects to the /video endpoint and connects it up to an image tag; https://github.com/autorope/donkeycar/blob/b1af0aacf488c938231912f89c4bfa5c2b11c649/donkeycar/parts/web_controller/templates/base_fpv.html#L72

So we are not using the video tag in the html, so this is probably why we don't automatically stay awake. If we can server those images as a true video then we can probably stay awake. We may also get better performance.

Ezward commented 3 years ago

All the code I have found for streaming video with Python does it the way we already do it; open a multi-part connectino and write jpegs to a img tag. Here is some nodejs code to stream an mp4 file to a video tag; it's pretty simple if you already have the data in an streaming video file. We need to do a live stream to the video tag.

Ezward commented 3 years ago

MDN page on live streaming; https://developer.mozilla.org/en-US/docs/Web/Guide/Audio_and_video_delivery/Live_streaming_web_audio_and_video . What we would need to do is to serve in a containerized video format in real-time. I don't find any native python libraries for this. There are a number that integrate with ffmpeg server on linux systems. There is also the OBS open source streaming server; https://obsproject.com/ So this all seems to require a complex integration of external servers. We would probably get much better performance using this than we currently get by writing jpegs to a stream.