search

Pedal-Intelligence / saypi-userscript

An independent voice interface for Inflection AI's conversational assistant, Pi
https://www.saypi.ai/
Other
15 stars 3 forks source link

Call fails to start after restore from submit error #78

Closed rosscado closed 3 months ago

rosscado commented 3 months ago

A user reported that after a session runs for a while, the call icon turns blue, indicating the service has ended unexpectedly. Upon further investigation, it was found that this issue occurs during the restore process after a submit error is handled.

Steps to Reproduce:

  1. Start a Say, Pi session and interact with Pi for some time.
  2. Trigger a submit error (exact steps to be determined, but likely involves Pi getting stuck and preventing message submission).
  3. Observe that Say, Pi detects the issue, creates a restore point, and reloads the page.
  4. After the page reloads and restores from the restore point, the call fails to start, and the call icon turns blue.

Expected Behavior: After restoring from a submit error, the call should resume seamlessly, and the call icon should turn grey to indicate an active session.

Actual Behavior: After restoring from a submit error, the call fails to start, and the call icon turns blue, indicating the service has ended.

call failed blue icon

Environment:

Additional Context: The issue appears to stem from the SubmitErrorHandler.ts module, specifically during the restore process. The checkForRestorePoint() method is responsible for restoring the application state, but it seems the call is not starting correctly after this restoration.

Relevant code snippet:

// In SubmitErrorHandler.ts
checkForRestorePoint(): void {
// ...
if (timeDifference <= 5) {
console.log("Restoring application state", restorePoint);
setFinalPrompt(restorePoint.prompt);
this.audioControls.activateAudioInput(restorePoint.audioInputEnabled);
this.audioControls.activateAudioOutput(restorePoint.audioOutputEnabled);
// Delete the executed restore point
localStorage.removeItem(this.restorePointKey);
 }
 }
rosscado commented 3 months ago

This is the console on reload during a restore:

Restoring application state 
{
  prompt: 'The... ',
  audioInputEnabled: true,
  audioOutputEnabled: true,
  creationTime: '2024-03-15T15:26:23.349Z'
}
saypi.user.js:26213 INFO: Final transcript: The... 
saypi.user.js:26206 DEBUG: Say, Pi Machine transitioned from inactive to callStarting with saypi:call
saypi.user.js:26508 Audio element found <audio>​</audio>​media
saypi.user.js:33061 desktop view
saypi.user.js:26788 Unable to exit full-screen mode. Maybe starting in desktop view?: Failed to execute 'exitFullscreen' on 'Document': Document not active (TypeError)
saypi.user.js:26206 DEBUG: Audio Output Machine transitioned from idle to loading with loadstart
_app-7b9d2df19f7d4cfa.js:11 DOMException: The play() request was interrupted by a new load request. https://goo.gl/LdLk22
(anonymous) @ _app-7b9d2df19f7d4cfa.js:11
Promise.catch (async)
(anonymous) @ 1850-1ac8bb69cb9ef00d.js:1
Rj @ framework-646a14d78de72d0b.js:9
Ik @ framework-646a14d78de72d0b.js:9
w @ framework-646a14d78de72d0b.js:9
J @ framework-646a14d78de72d0b.js:33
R @ framework-646a14d78de72d0b.js:33
saypi.user.js:26206 DEBUG: Audio Output Machine transitioned from loading to loaded:ready with loadedmetadata
saypi.user.js:26206 DEBUG: Audio Output Machine transitioned from loaded:ready to loaded:playing with play
saypi.user.js:26206 DEBUG: Audio Output Machine transitioned from loaded:playing to loaded:paused with pause
saypi.user.js:26206 DEBUG: Say, Pi Machine transitioned from callStarting to inactive with xstate.after(20000)#sayPi.callStarting
rosscado commented 3 months ago

Resolved in v1.5.12, along with some bonus goodies.

https://github.com/Pedal-Intelligence/saypi-userscript/assets/16578183/c240db13-0264-4470-97b8-7e94d68d225b