k2-fsa / sherpa-onnx

Speech-to-text, text-to-speech, speaker diarization, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kotlin, C#, Go, NodeJS, Java, Swift, Dart, JavaScript, Flutter, Object Pascal, Lazarus, Rust
https://k2-fsa.github.io/sherpa/onnx/index.html
Apache License 2.0
3.69k stars 428 forks source link

Flutter Isolate #1530

Closed germes96 closed 2 weeks ago

germes96 commented 2 weeks ago

The offline transcriber works very well when run on the ui Thread, but it blocks the interface.

When run in isolation or compute mode the result is always empty.


  SendPort sendPort = data[0];
  final String filePath = data[1]; //0 is for FileName
  logger.i('THE REVEIVE AUDIO PATH ${filePath}');
  final sherpa_onnx.OfflineRecognizer? _recognizer = data[2]; //0 is for FileName

  final waveData = sherpa_onnx.readWave(filePath);
  final stream = _recognizer?.createStream();
  stream?.acceptWaveform(samples: waveData.samples, sampleRate: waveData.sampleRate);

  final startTime = DateTime.now();
  _recognizer?.decode(stream!);
  var result = _recognizer?.getResult(stream!);

  final endTime = DateTime.now();
  final duration = endTime.difference(startTime).inMilliseconds;
  logger.i('Task duration: $duration milliseconds, data: ${result?.text}');
  stream?.free();
  sendPort.send(result?.text ?? 'BAD ASR');
}

```Future<String> transcribeInCompute(List<dynamic> data) async {
  final String filePath = data[0]; //0 is for FileName
  logger.i('THE REVEIVE AUDIO PATH ${filePath}');
  final sherpa_onnx.OfflineRecognizer? _recognizer = data[1]; //0 is for FileName

  final waveData = sherpa_onnx.readWave(filePath);
  final stream = _recognizer?.createStream();
  stream?.acceptWaveform(samples: waveData.samples, sampleRate: waveData.sampleRate);

  final startTime = DateTime.now();
  _recognizer?.decode(stream!);
  var result = _recognizer?.getResult(stream!);

  final endTime = DateTime.now();
  final duration = endTime.difference(startTime).inMilliseconds;
  logger.i('Task duration: $duration milliseconds, data: ${result?.text}');
  stream?.free();
  return result?.text ?? 'BAD ASR';
}
csukuangfj commented 2 weeks ago

could you check that _recognizer.ptr is not 0?

Also, make sure you have called init sherpa onnx binding in the isolate.

germes96 commented 2 weeks ago

Thanks. The solution was to init sherpa_onnx binding in the isolate. Now the ptr is good.