search

h2oai / wave

Realtime Web Apps and Dashboards for Python and R
https://wave.h2o.ai
Apache License 2.0
4k stars 327 forks source link

Components to use microphone / camera #1459

Open vopani opened 2 years ago

vopani commented 2 years ago

Is your feature request related to a problem? Please describe

With some fantastic new components like text and image annotators enabling seamless data science workflows, it would be cool to take it one step further by using microphones and cameras.

Describe the solution you'd like

Components that can capture audio or video using available microphone / camera device from directly inside an app and make the audio / video files available for the app for processing / inferencing.

Additional context

This would greatly benefit some of the deep learning apps on audio/video projects, especially Automatic Speech Recognition, Speech-To-Text, Face/Gesture Recognition, Video Inferencing, etc.

cc: @leecming82

mturoci commented 2 years ago

Currently, Wave serializes all the data and sends them up the websocket (q.args/q.events). The main work here is to allow for sending binary (video/audio) data as well.

vopani commented 2 years ago

Possible with some JS using: https://medium.com/@unusualcode/record-audio-in-h2o-wave-apps-fcd6c0b05e59 Thanks @mturoci !