search

tareq21-meet / Artemis

0 stars 0 forks source link

Realtime listening and speech-to-text #1

Open tombler opened 3 years ago

tombler commented 3 years ago

The problem:

Recording audio needs to start in realtime (or as close as possible). As soon as a sexual assault keyword is heard, the recording needs to be triggered so that no evidence is lost.

Solution breakdown:

  1. Convert small sections of audio (how much? 10 sec? 5 sec?) into text
  2. Check the converted text for a list of sexual assault keywords (where does this list come from?)

Tools:

https://developer.android.com/reference/android/speech/RecognitionListener#onPartialResults(android.os.Bundle) (@tareq21-meet is this the function that you're using?)

tombler commented 3 years ago

@tareq21-meet I have a few suggestions. It sounds like you're having trouble with the onPartialResults function and with getting realtime text-to-speech to work in general. There are definitely technical solutions that could work inside your own Android app, but implementing them might be too complicated to get working in your timeframe. For example, Google's speech-to-text API is very robust, but setting it up requires knowledge of how Google Cloud works and might not be realistic for your timeline. Here are some links anyway:

https://cloud.google.com/speech-to-text/docs/streaming-recognize#speech-streaming-recognize-java https://cloud.google.com/speech-to-text/docs/endless-streaming-tutorial

My suggestion

Instead of trying to get the realtime speech-to-text working inside of your app, you can use your code to trigger Android opening another app that already does realtime speech-to-text, namely Live Transcribe https://play.google.com/store/apps/details?id=com.google.audio.hearing.visualization.accessibility.scribe. It's free to download, and you can require that users have that app downloaded alongside yours. The flow could look something like this:

  1. Make sure the user has Live Transcribe installed on their Android phone.
  2. Your code recognizes a keyword.
  3. On recognition, send a command for Android to open Live Transcribe. Usually when you send a system command to open an app, you can also send a command for what to do when the app opens. In this case, you'd want to tell Live Transcribe to "start transcribing" as soon as it's open. All of this would happen automatically without the user having to open the app themselves. See this Stack Overflow answer: https://stackoverflow.com/a/50347982/6373055.

Something like this (this is pseudocode, not tested) would automatically open up the Live Transcribe app for the user. You'd have to figure out how to tell Live Transcribe to automatically start transcribing when it opens.

// Google Play URL: https://play.google.com/store/apps/details?id=com.google.audio.hearing.visualization.accessibility.scribe
// app ID = com.google.audio.hearing.visualization.accessibility.scribe

Intent intent = context.getPackageManager().getLaunchIntentForPackage("com.google.audio.hearing.visualization.accessibility.scribe");
intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
startActivity(intent);

Obviously this is an MVP solution - in an ideal world, you wouldn't want a user to have to download two apps (yours + Live Transcribe), but given your timeline, this may be a good proof-of-concept. Let me know if this helps.