VAD is a Flutter library for Voice Activity Detection (VAD) across iOS , Android , and Web platforms. This package allows applications to start and stop VAD-based listening and handle various VAD events seamlessly.
Under the hood, the VAD Package uses dart:js_interop for Web to run VAD JavaScript library and onnxruntime for iOS and Android utilizing onnxruntime library with full-feature parity with the JavaScript library.
The package provides a simple API to start and stop VAD listening, configure VAD parameters, and handle VAD events such as speech start, speech end, errors, and misfires.
Check out the VAD Package Example App to see the VAD Package in action on the Web platform.
Cross-Platform Support: Works seamlessly on iOS, Android, and Web.
Event Streams: Listen to events such as speech start, speech end, errors, and misfires.
Configurable Parameters: Customize VAD behavior with various parameters.
Easy Integration: Simple setup and usage within Flutter applications.
Before integrating the VAD Package into your Flutter application, ensure that you have the necessary configurations for each target platform.
To use VAD on the web, include the following scripts in your web/index.html file to load the necessary VAD libraries:
<!-- VAD Dependencies -->
<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/ort.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@ricky0123/vad-web@0.0.19/dist/bundle.min.js"></script>
<script src="https://cdn.jsdelivr.net/gh/keyur2maru/vad_dart@latest/dist/vad_web.js"></script>For iOS, you need to configure microphone permissions and other settings in your Info.plist file.
ios/Runner/Info.plist and add the following entries to request microphone access:<key>NSMicrophoneUsageDescription</key>
<string>This app needs access to the microphone for Voice Activity Detection.</string>Podfile includes the necessary build settings for microphone permissions:post_install do |installer|
installer.pods_project.targets.each do |target|
flutter_additional_ios_build_settings(target)
target.build_configurations.each do |config|
config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] ||= [
'$(inherited)',
'PERMISSION_MICROPHONE=1',
]
end
end
endFor Android, configure the required permissions and build settings in your AndroidManifest.xml and build.gradle files.
android/app/src/main/AndroidManifest.xml and add the following permissions:<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>android/app/build.gradle and add the following settings:
android {
compileSdkVersion 34
...
}Add the VAD Package to your pubspec.yaml dependencies:
Note: audioplayers is used for audio playback in the example below. You can replace it with any other audio player package of your choice.
dependencies:
flutter:
sdk: flutter
vad: ^0.0.4
permission_handler: ^11.3.1
audioplayers: ^6.1.0Then, run flutter pub get to fetch the packages.
Below is a simple example demonstrating how to integrate and use the VAD Package in a Flutter application. For a more detailed example, check out the VAD Example App in the GitHub repository.
// main.dart
import 'package:flutter/material.dart';
import 'package:permission_handler/permission_handler.dart';
import 'package:vad/vad.dart';
void main() async {
WidgetsFlutterBinding.ensureInitialized();
runApp(const MyApp());
}
class MyApp extends StatelessWidget {
const MyApp({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
home: Scaffold(
appBar: AppBar(title: const Text("VAD Example")),
body: const MyHomePage(),
),
);
}
}
class MyHomePage extends StatefulWidget {
const MyHomePage({super.key});
@override
State<MyHomePage> createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
final _vadHandler = VadHandler.create(isDebug: true);
bool isListening = false;
final List<String> receivedEvents = [];
@override
void initState() {
super.initState();
_setupVadHandler();
}
void _setupVadHandler() {
_vadHandler.onSpeechStart.listen((_) {
debugPrint('Speech detected.');
setState(() {
receivedEvents.add('Speech detected.');
});
});
_vadHandler.onSpeechEnd.listen((List<double> samples) {
debugPrint('Speech ended, first 10 samples: ${samples.take(10).toList()}');
setState(() {
receivedEvents.add('Speech ended, first 10 samples: ${samples.take(10).toList()}');
});
});
_vadHandler.onVADMisfire.listen((_) {
debugPrint('VAD misfire detected.');
setState(() {
receivedEvents.add('VAD misfire detected.');
});
});
_vadHandler.onError.listen((String message) {
debugPrint('Error: $message');
setState(() {
receivedEvents.add('Error: $message');
});
});
}
@override
void dispose() {
_vadHandler.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) {
return Padding(
padding: const EdgeInsets.all(16),
child: Column(
mainAxisSize: MainAxisSize.min,
children: [
ElevatedButton.icon(
onPressed: () async {
setState(() {
if (isListening) {
_vadHandler.stopListening();
} else {
_vadHandler.startListening();
}
isListening = !isListening;
});
},
icon: Icon(isListening ? Icons.stop : Icons.mic),
label: Text(isListening ? "Stop Listening" : "Start Listening"),
style: ElevatedButton.styleFrom(
minimumSize: const Size(double.infinity, 48),
),
),
const SizedBox(height: 8),
TextButton.icon(
onPressed: () async {
final status = await Permission.microphone.request();
debugPrint("Microphone permission status: $status");
},
icon: const Icon(Icons.settings_voice),
label: const Text("Request Microphone Permission"),
style: TextButton.styleFrom(
minimumSize: const Size(double.infinity, 48),
),
),
const SizedBox(height: 16),
Expanded(
child: ListView.builder(
itemCount: receivedEvents.length,
itemBuilder: (context, index) {
return ListTile(
title: Text(receivedEvents[index]),
);
},
),
),
],
),
);
}
}VadHandler with debugging enabled.onSpeechStart, onSpeechEnd, onError, onVADMisfire).Permissions:
Listening Controls:
createCreates a new instance of the VadHandler with optional debugging enabled with the isDebug parameter and optional configurable model path with the modelPath parameter but it's only applicable for the iOS and Android platforms. It has no effect on the Web platform.
startListeningStarts the VAD with configurable parameters. Note: The sample rate is fixed at 16kHz, which means one frame is equal to 1536 samples or 96ms.
void startListening({
double positiveSpeechThreshold = 0.5,
double negativeSpeechThreshold = 0.35,
int preSpeechPadFrames = 1,
int redemptionFrames = 8,
int frameSamples = 1536,
int minSpeechFrames = 3,
bool submitUserSpeechOnPause = false,
});stopListeningStops the VAD session.
void stopListening();disposeDisposes the VADHandler and closes all streams.
void dispose();onSpeechEndTriggered when speech ends with optional audio data.
Stream<List<double>> get onSpeechEnd;onSpeechStartTriggered when speech starts.
Stream<void> get onSpeechStart;onVADMisfireTriggered when the VAD misfires.
Stream<void> get onVADMisfire;onErrorTriggered when an error occurs.
Stream<String> get onError;Proper handling of microphone permissions is crucial for the VAD Package to function correctly on all platforms.
Configuration: Ensure that NSMicrophoneUsageDescription is added to your Info.plist with a descriptive message explaining why the app requires microphone access.
Runtime Permission: Request microphone permission at runtime using the permission_handler package.
Configuration: Add the RECORD_AUDIO, MODIFY_AUDIO_SETTINGS, and INTERNET permissions to your AndroidManifest.xml.
Runtime Permission: Request microphone permission at runtime using the permission_handler package.
To prevent memory leaks and ensure that all resources are properly released, always call the dispose method on the VadHandler instance when it's no longer needed.
vadHandler.dispose();The VAD Package has been tested on the following platforms:
Contributions are welcome! Please feel free to submit a pull request or open an issue if you encounter any problems or have suggestions for improvements.
Special thanks to Ricky0123 for creating the VAD JavaScript library, gtbluesky for building the onnxruntime package and Silero Team for the VAD model used in the library.
This project is licensed under the MIT License . See the LICENSE file for details.
For any issues or contributions, please visit the GitHub repository .