search

denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
94.07k stars 5.23k forks source link

Feature Request: Support Web Media Capture and Streams API #16312

Open h76oeI6pMxU9g4p8aCpc6Q opened 1 year ago

h76oeI6pMxU9g4p8aCpc6Q commented 1 year ago

I can see there should have many use cases will be opened if this Web API is usable on the server side https://developer.mozilla.org/en-US/docs/Web/API/Media_Capture_and_Streams_API https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia

E.g. Use cases: OpenCV like Real-time image recognition, face recognition on streaming video from webcam on server side..... Training AI model using WebGPU........ Real-time QR code recognition May be use with std/collections API to analyze the large 2D array????

Must ask for camara/mic permission before Deno can access them. Deno can open a small window on server side along with the original CLI to displaying the video stream to user like this.

opencv_tutorial_header

What I mentioned above are just some very basic ideas, not standard descriptions, and do not need to be used as a necessary reference.

Currently, I am not aware of any other back-end JavaScript/Typescript runtime environment (Node.js or Bun) that implements this Web API, and if Deno does it first, it will be a revolutionary move.

This is very encouraging.

crowlKats commented 1 year ago

The getUserMedia API might be problematic, as user prompts for APIs have been discussed in the past for other APIs but never came to a proper solution.

Deno can open a small window on server side along with the original CLI to displaying the video stream to user like this.

No, we are not going to implement a windowing solution in deno just for this; also how would that work on headless system?

h76oeI6pMxU9g4p8aCpc6Q commented 1 year ago

No, we are not going to implement a windowing solution in deno just for this; also how would that work on headless system?

Deno could detect the system condition. This Web API should only usable on non-headless system such as Windows, macOS and Linux Destop.

crowlKats commented 1 year ago

No, an API has to be headless for us to support it. Non-headless APIs have already gotten denied even if simple or small (clipboard, notification). Also, as stated, we will not include a windowing system in deno. The implementation would only cover parts of the API that would be possible to use in a headless context.

h76oeI6pMxU9g4p8aCpc6Q commented 1 year ago

@crowlKats This is just an idea sharing. I also think don't need to support this on near future. I suggest to put this post to the "future" milestone.

This Web API may be a planned feature of Deno v50 (current is Deno v1.26.1), nobody knows.

jqhr commented 1 year ago

I need it too! For example, develop face detection app on Raspberry PI.