Maclean-D / three-assistant-glass

Customizeable 3D conversational AI character with Vapi, three-vrm, & Looking Glass support.
GNU Affero General Public License v3.0
11 stars 2 forks source link

Clipboard Access fails when non-plaintext is copied #1

Open Maclean-D opened 2 months ago

Maclean-D commented 2 months ago

Copying non-plaintext when Clipboard Access is on throws this error:

Error: Command failed with exit code 101: C:\Users\mac\Documents\GitHub\three-assistant-glass\node_modules\clipboardy\fallbacks\windows\clipboard_x86_64.exe --paste
thread 'main' panicked at 'Error: Could not paste from clipboard: Error { repr: Os { code: 1168, message: "Element not found." } }', src\libcore\result.rs:906:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.

    at makeError (file:///C:/Users/mac/Documents/GitHub/three-assistant-glass/node_modules/execa/lib/error.js:60:11)
    at handlePromise (file:///C:/Users/mac/Documents/GitHub/three-assistant-glass/node_modules/execa/index.js:124:26)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.paste (file:///C:/Users/mac/Documents/GitHub/three-assistant-glass/node_modules/clipboardy/lib/windows.js:16:20) {
  shortMessage: 'Command failed with exit code 101: C:\\Users\\mac\\Documents\\GitHub\\three-assistant-glass\\node_modules\\clipboardy\\fallbacks\\windows\\clipboard_x86_64.exe --paste',
  command: 'C:\\Users\\mac\\Documents\\GitHub\\three-assistant-glass\\node_modules\\clipboardy\\fallbacks\\windows\\clipboard_x86_64.exe --paste',
  escapedCommand: '"C:\\Users\\mac\\Documents\\GitHub\\three-assistant-glass\\node_modules\\clipboardy\\fallbacks\\windows\\clipboard_x86_64.exe" --paste',
  exitCode: 101,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: `thread 'main' panicked at 'Error: Could not paste from clipboard: Error { repr: Os { code: 1168, message: "Element not found." } }', src\\libcore\\result.rs:906:4\n` +
    'note: Run with `RUST_BACKTRACE=1` for a backtrace.\n',
  cwd: 'C:\\Users\\mac\\Documents\\GitHub\\three-assistant-glass',
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false
}

Possible solutions from most to least favorable:

1. Add support for text and images

When plaintext or an image is copied, send it to Vapi. If something else is copied throw an alert that says that file type isn't supported.

2. Process images with Google Vision API

When an image is copied send to Google Vision for a text description, then send the text description to Vapi. Google Vision for images should also be optional. Ideally only resort to this if Vapi won't take images.

3. Only support plaintext

If non-plaintext is copied throw an alert that says it's not supported and don't try to process it or send it anywhere. Ideally only resort to this if there's no clipboard library that supports images.