search

Pedal-Intelligence / saypi-userscript

An independent voice interface for Inflection AI's conversational assistant, Pi
https://www.saypi.ai/
Other
15 stars 4 forks source link

Pi Responding to Self in Immersive Mode (Android, Possibly Environment-Specific) #81

Closed rosscado closed 3 days ago

rosscado commented 8 months ago

A user (DawidM) reported that Pi sometimes responds to its own statements while using the Immersive Mode feature on Android (Kiwi browser). This creates a self-response loop.

User Feedback:

Device: Android phone (unspecified model) Browser: Kiwi Browser (version unknown) Reproducibility: Issue occurs in Immersive Mode. Unknown if it persists in regular/text mode. Potential Causes:

Audio reverberation: In Immersive Mode, the phone's speaker and microphone are in close proximity. Pi's audio output may reverberate and be picked up by the microphone, triggering unintended responses. Android audio handling: Android's audio handling may differ from other platforms and contribute to audio feedback. Browser-specific issue: Kiwi Browser's handling of audio streams could be a factor. Troubleshooting Steps:

Gather more data: Asked other Android users in our community if they experience similar issues. Collect device, browser, and headphone/speaker-mode details. Investigate environment: If possible, ask DawidM to test in different environments (quiet vs. noisy, with headphones vs. without) to see if reverberation is the root cause. Explore temporary workaround: Consider adding a user setting to temporarily disable microphone input during Pi's speech output. This could mitigate the issue until the root cause is identified. Or instruct users to wear headphones (not ideal). Additional Notes:

The upcoming "interrupt Pi" feature could worsen this issue. Understanding the cause and potential solutions is important.

rosscado commented 7 months ago

Gavin also reported, and we reproduced this issue on desktop when the audio output and microphone input devices are separated. E.g. when using HomePod as speaker and monitor as microphone, the microphone picks up the speaker's output.

rosscado commented 6 months ago

It's notable that DawidM experiences both this issue and #58. Given what we saw with Gavin and our Homepod test, this suggests both the self-talk and audio quality/media volume issues are related to microphone positioning/performance.

rosscado commented 3 days ago

To address the issue of Pi responding to its own statements due to ineffective echo cancellation on some devices (such as certain browsers and Android devices), we recommend disabling the automatic hands-free interrupt feature. This will stop the feedback problem and provide a manual tap-to-interrupt feature instead. This solution is available starting from Say, Pi version 1.6.5.

rosscado commented 3 days ago

In future versions we should consider trying to automatically detect this feedback behaviour and prompt the user with the recommended configuration change, or apply it for them. Preliminary attempts to detect this behaviour have proven unsuccessful (see AudioCapabilities.ts).