search

ml-explore / mlx-examples

Examples in the MLX framework
MIT License
6.3k stars 897 forks source link

[Feature Request] Support Speaker Diarization #1039

Open uniqueness-ae opened 1 month ago

uniqueness-ae commented 1 month ago

Implement speaker diarization for the existing mlx whisper support to:

  1. Enhance transcription accuracy in multi-speaker conversations
  2. Distinguish between different speakers in the output
  3. Improve overall usability of the transcription feature

This addition will provide more insightful and structured transcripts, making it easier to analyze and understand complex audio content. Thanks

Hoohm commented 1 month ago

Would love to see this as well. I can help out in making the feature but I need some pointers as to how it would be possible.

uniqueness-ae commented 1 month ago

I tried pyannote.audio model using rented cloud GPUs and had some success. Perhaps if there is a way to run this mlx, it will probably run faster. Maybe even better if it’s coupled with whisper to simplify the process. There is a repo from m-bain called WhisperX that does this. Could help as a reference.