search

JamesNewton / AdvancedRoboticsWithJavascript

A course in Robotics programming using Javascript in the Dexter Development Environment
The Unlicense
6 stars 8 forks source link

Problems with Camera Access from page served via ESP-8266 #76

Open JamesNewton opened 4 years ago

JamesNewton commented 4 years ago

We were not able to get reliable access to the smartphone camera from a web page served by the bot. https://github.com/JamesNewton/AdvancedRoboticsWithJavascript/wiki#image-subtraction-in-a-smartphone-browser

Part of this was because we used an unreliable method of accessing the camera, but another method was found: https://googlechrome.github.io/samples/image-capture/grab-frame-take-photo.html and the code has been updated to support it: https://jsfiddle.net/4r8u1n37/2/

Either method uses navigator.mediaDevices.getUserMedia({video: true}) but the better way adds to the video promise:

    const track = mediaStream.getVideoTracks()[0];
    imageCapture = new ImageCapture(track);

and then uses a imageCapture.grabFrame().then(frame => { ... }) promise to grab the frames.

Another issue was that because the robot can only serve web pages via HTTP (non-secure) the Chrome web browser will not allow access to the camera. It should have been possible to override this limitation (it is in the PC)

But on the Android version of Chrome, the URL to treat as secure was not retained when restarting the browser app: https://stackoverflow.com/questions/52944873/android-how-to-set-unsafely-treat-insecure-origin-as-secure-http-example-co

This is apparently by design because the phone user doesn't have root access on a non-hacked phone.

So it looks like this /might/ be an answer if we can squeeze it into a bot, https://github.com/me-no-dev/ESPAsyncWebServer/issues/75

JamesNewton commented 4 years ago

If we can upgrade to the ESP32 in future classes, then we can do things like this: https://www.youtube.com/watch?v=dZrCx_RIdMI Note that this uses the camera on the ESP32, and serves the tenserflow-light js file from the internet, which then requires the robot to be on the local router instead of being an access point, but could easily be updated to use the camera in the smartphone.