pipecat-ai / pipecat

Open Source framework for voice and multimodal conversational AI
BSD 2-Clause "Simplified" License
3.47k stars 341 forks source link

Add STTMuteFilter to un/mute the STT #716

Closed markbackman closed 2 weeks ago

markbackman commented 2 weeks ago

Please describe the changes in your PR. If it is addressing an issue, please reference that as well.

This adds a new processor called STTMuteFilter, which can mute the STT service while the bot is speaking. In this context muting means:

This processor is intended to be added to the pipeline before the STT service.

To help accomplish this task, I've added a new frame type, STTMuteFrame, which is emitted by the STTMuteFilter based on the processor settings.

The processor can be configure to:

I've included foundational example 24-stt-mute-processor.py to show how to use this.

markbackman commented 2 weeks ago

@aconchillo I'm not sure how to classify this processor. I stuck it with filters but that doesn't feel right.

aconchillo commented 2 weeks ago

This is awesome! Just added a few comments.

markbackman commented 2 weeks ago

Ok! All changes are made. Do you think this belongs in pipecat.processors.filters or somewhere else?

aconchillo commented 2 weeks ago

Ok! All changes are made. Do you think this belongs in pipecat.processors.filters or somewhere else?

Thanks! That's an always hard question. If it was a filter we would call it stt_mute_filter.py in which case the class would be STTMuteFilter. The concept of a filter is to stop or passthrough certain frames, and that's what this processor is doing, so it might make sense to have it there, so we should rename it STTMuteFilter probably as all the rest.