Add "capture" variant code for Raspberry Pi camera streaming

[jywarren]

This is an example of how a webpage running on a Raspberry Pi (using code compiled via http://publiclab.org/pi-builder, for example), with a camera attached, can stream camera video directly into a WebRTC app like this one:

https://github.com/publiclab/infragram/blob/f558241119145a0b1246832a2e86c7d75268c71b/pi/index.html#L185-L191

Let's port this into an alternative version of the capture code (possibly after refactoring the camera code as in #16 and #56), so that people can connect to their spectrometers via a web interface easily.

The capture code is here, but we could make an alternative one named pi.html:

https://github.com/publiclab/spectral-workbench.js/blob/main/examples/capture/index.html

[jywarren]

Would you like to open a new PR which adds a button (labelled "Connect to Raspberry Pi") to the start screen, using the following code?

https://github.com/publiclab/infragram/blob/main/pi/index.html#L185-L191

We'll have to adjust this line to correctly send the video data from a raspberry pi camera to the spectral Workbench code:

infragram.options.processor.updateImage(piImage);

We can make a new function that inserts the video image frame into the spectral workbench capture code, which is also based around a canvas element. We can use code kind of like this:

https://github.com/publiclab/infragram/blob/de33d2af07e7affa60584f7d42bcb111f4036e7b/src/processors/javascript.js#L224

Actually that whole function is pretty good and could be copied over:

function updateImage(img) {
    var ctx, height, imgCanvas, width;
    imgCanvas = document.getElementById("image"); // this will have to be modified for a unique id
    ctx = imgCanvas.getContext("2d");
    width = img.videoWidth || img.width;
    height = img.videoHeight || img.height;
    ctx.drawImage(img, 0, 0, width, height, 0, 0, imgCanvas.width, imgCanvas.height);
  }

From capture.js, I think we can simply directly use the canvas element in the demo code, but we'll have to give it a unique ID we can address to get the above line to correctly link with it.

Does this make sense? Try it out in a PR! It'll be tough to test but we can work on that together.

[jywarren]

If you get stuck because you don't have a pi camera, what you can do is just point it at a local image instead of the video source. So instead of this:

          piImage.src = "http://pi.local/cam/cam_pic.php?time=" + new Date().getTime();

You can just point at a local image:

          piImage.src = "path/to/image.png";

Then that'll let you test the code!

[sidntrivedi012]

@jywarren Do I have to add the button to the example page of capture or on the start screen of spectralworkbench webapp?

[jywarren]

Just the capture screen!

On Tue, Jul 16, 2019, 2:59 PM Siddhant N Trivedi notifications@github.com wrote:

@jywarren https://github.com/jywarren Do I have to add the button to the example page of capture or on the start screen of spectralworkbench webapp?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/spectral-workbench.js/issues/75?email_source=notifications&email_token=AAAF6JZD7WNICUV3MWNHTLDP7YK5ZA5CNFSM4GSS56AKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2B2DVQ#issuecomment-511943126, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6J5AJZGE36LBWEEGT33P7YK5ZANCNFSM4GSS56AA .

[jywarren]

Like here, maybe next to "camera selection"?

[sidntrivedi012]

Sure, thanks @jywarren. Adding it.

[sidntrivedi012]

[jywarren]

Awesome, thanks!

On Tue, Jul 16, 2019, 3:13 PM Siddhant N Trivedi notifications@github.com wrote:

Sure, thanks @jywarren https://github.com/jywarren. Adding it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/spectral-workbench.js/issues/75?email_source=notifications&email_token=AAAF6JZILKUW372N47M52HTP7YMUZA5CNFSM4GSS56AKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2B3JTA#issuecomment-511947980, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6J3YKCO7JSE5AEPX52TP7YMUZANCNFSM4GSS56AA .

[sidntrivedi012]

[sidntrivedi012]

@jywarren Which one looks better?

[starkblaze01]

@jywarren Which one looks better?

😅The second one looks better but it more sense to keep it along with selecting a camera.

[jywarren]

Hm maybe we make it not a priority button and it'll not call as much attention as the "start" button... But yeah I think I agree with Mayank. Great!

[sidntrivedi012]

Thanks @starkblaze01. Made a PR #103 to make sure that this issue is solved in it.

[sidntrivedi012]

@jywarren In order to add the infragram code into the capture part, I need to add infragram-js. Should I directly copy it from https://github.com/publiclab/infragram/blob/main/dist/infragram.js ?

[starkblaze01]

@jywarren In order to add the infragram code into the capture part, I need to add infragram-js. Should I directly copy it from https://github.com/publiclab/infragram/blob/main/dist/infragram.js ? @sidntrivedi012 No, it is already published as node module package. Simply do npm i infragram And import it then. I think that will work

[sidntrivedi012]

Sure @starkblaze01 . Thanks. 🙂

[jywarren]

Hi all! I'll leave comments in the PR here as it'll be easier to note lines of code: https://github.com/publiclab/spectral-workbench.js/pull/103 Thanks!

[jywarren]

I think this would do to get a Pi running:

var piImage = new Image();
$W.getCrossSection = function() {
  $W.ctx.drawImage(piImage, 0, -$W.sample_start_row);
  piImage.src = "http://pi.local/cam/cam_pic.php?time=" + new Date().getTime(); 
}

This could be seriously improved by getting the preview window working. But it'll do OK for starters. I think the images could be drawn to the <video> element too... but all this is a follow-up issue, don't worry about this just yet.

Originally posted by @jywarren in https://github.com/publiclab/spectral-workbench.js/issues/56#issuecomment-438465614