access to browser getUserMedia() via Electron's desktopCapturer

[elrypto]

Describe the solution you'd like and the expected behavior

Integrate Electron's desktopCapturer, documentation here: https://electronjs.org/docs/api/desktop-capturer.

Allow access to the users desktop audio and video with the following parameters supported (as described in documentation):

To capture both audio and video from the entire desktop the constraints passed to navigator.mediaDevices.getUserMedia must include chromeMediaSource: 'desktop', for both audio and video, but should not include a chromeMediaSourceId constraint.

Is your feature request related to a problem? Please describe.

Currently, a call to navigator.mediaDevices.getUserMedia() fails without an alternative.

Possible implementation / References

https://electronjs.org/docs/api/desktop-capture

Additional context

[mhuggins]

Thanks for submitting this!

[elrypto]

Thanks for suggesting Matt - will give it a go over the next couple of weeks (will probably have a couple of questions).

On Sun, Jul 29, 2018 at 11:15 AM, Matt Huggins notifications@github.com wrote:

Thanks for submitting this!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nos/client/issues/375#issuecomment-408684502, or mute the thread https://github.com/notifications/unsubscribe-auth/AlQda1yPLdYy0jmctewuOqBHngGInPfEks5uLdF3gaJpZM4VlcUW .

[mhuggins]

It seems like we may not have the option to make this work. According to this article:

Browser vendors have recently ruled that getUserMedia should only work on https: protocol. You'll need a SSL certificate for this API to work.

As an example, loading https://davidwalsh.name/demo/camera.php works within nOS because it's an https URL.

[mhuggins]

It looks like we can register a protocol as "secure" in electron. So in theory, we could change this line:

https://github.com/nos/client/blob/321dbf26928380bc0b6affb0aca70830f0a3c053/src/main/index.js#L19

to instead read:

protocol.registerStandardSchemes(['nos'], { secure: true });

The problem is that I don't think we can actually consider a secure scheme in a security sense. As far as I understand, the RPC servers aren't using Secure RPC, and many of the consensus nodes run on HTTP rather than HTTPS as well.

[mhuggins]

@deanpress I think we should opt to close this and #345 given the above, but I wanted to see what you think. Users will still be able to use getUserMedia via HTTPS within the client as they can today.

[mhuggins]

A possible solution for content hosted coming from the nos protocol would be to communicate with an IFRAME that's using the https protocol via cross-origin communication.

[elrypto]

Thanks for looking into this Matt, appreciate the notes, you saved me a lot of time.

From the messages, the suggested approach right now would be use an iframe and make the getusermedia() call within the iframe going to a https address. I will give that a go.

Probably worth considering a secure protocol (nos secure) in the future, with most public websites going to https by default now.

Regards, Lloyd

On Sun, Sep 23, 2018 at 7:32 PM Matt Huggins notifications@github.com wrote:

A possible solution for content hosted coming from the nos protocol would be to communicate with an IFRAME that's using the https protocol via cross-origin communication https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nos/client/issues/375#issuecomment-423856247, or mute the thread https://github.com/notifications/unsubscribe-auth/AlQda5Qq-PENa9M6NBCvliEs6bri9Y1Uks5ueBomgaJpZM4VlcUW .

[jeroenptrs]

The problem is that I don't think we can actually consider a secure scheme in a security sense. As far as I understand, the RPC servers aren't using Secure RPC, and many of the consensus nodes run on HTTP rather than HTTPS as well.

@mhuggins what about proxying the consensus servers with a https server? Similar to this https://github.com/be-neo/neo-https-proxy

[mhuggins]

@jeroenptrs Interesting idea. Where is this being used today? Would it integrate well with neon-js by chance?

[DalderupMaurice]

Good old https proxy days hahaha amazing work @jeroenptrs 💪 we've used this on NeoBlog because IPFS requires a secure connection and the NEO nodes require a connection over HTTP. Due to this, we got a mixed content issue & Jeroen fixed it with this proxy module, to enable interactions with the IPFS nodes @mhuggins

[mhuggins]

If we can make this work with neon-js, then that might be a good short-term solution. The downside is that it's a centralized solution.

It seems like this is something that should be solved by NGD / added to NEO core long term. Any reason it shouldn't be that you guys know of?

[jeroenptrs]

If you start off http, it's gonna be hard to enforce https on every node. Especially since it requires reverse proxies and the whole #!, but NGD could force it on nodes... don't think they will though