Extract keywords - error sending image

[nigelp]

Hi, getting weird one. Error sending image on the extract keyword setting. On WiFi on Samsung S20 FE Android 13.

All works well otherwise I think. :)

[alexdredmon]

@nigelp thank you for reporting this! That sounds like a possible intermittent connectivity issue, especially if you're experiencing it only on that prompt and not others.

Can you please try the following?

• Confirm whether other prompts (e.g. "What's this made of?") work correctly
• Try the extract keywords prompt again to see if the issue was intermittent
• Connect to a different WiFi network or use cellular connection
• Ensure you have the latest version of the app and/or reinstall it

If none of these options work, just let me know and I can add additional logging to debug/fix.

[nigelp]

@alexdredmon thanks for the response. Hope I can help.

Tried as follows with no luck:

1. With 4G/5G - no wifi
2. Uninstalled and reinstalled
3. Tried all the prompts
4. Exact Keywords again, no joy.

I am attaching two screengrabs.

One time it DID work was with where am I when out and about in a garden. It gave a full answer with location (scary spooky - heh). That must have been on 5G I guess.

One other point to note, there's no recognition that it's saving my OpenAI API key. Just goes blank in the field when I click Save.

I really want this to work, because I think it'll be spectacular when it does. :)

[alexdredmon]

Thank you for the detailed info and screenshots - this is all helpful.

Are you able to hit the following URL on the same network/configuration that you were getting those "Handshake Exception" errors? URL: https://api.openai.com/

It should return something like:

{
"message": "Welcome to the OpenAI API! Documentation is available at https://platform.openai.com/docs/api-reference"
}

I'll add some additional debug logging to try to help track down the other nondescript error as well.

[nigelp]

Yep, it displayed the right OpenAI message. :)

[nigelp]

Hi there, any news? :)

[alexdredmon]

@nigelp I've added some logic and logging around this since your last report / screenshots - please let me know if you're still seeing any ongoing issues and if so what specific error(s) you're getting (as they should be more verbose now).

Also if you're still having problems it might be worth trying to configure a local model via the newly supported custom engine feature - here's a rundown in the README: https://github.com/alexdredmon/crayeye/?tab=readme-ov-file#local-models

[nigelp]

@alexdredmon oh cool, the original error has gone away. Now I'm just getting an image size error, see attached. I don't think Samsung phones take images which are bigger than 20MB actually (I think they're more like 2MB), so not sure what's going on. :)

[alexdredmon]

@nigelp interesting - would you be able to take a photo with your phone and upload it in this thread for debugging please?

Another thing you might try is toggling HDR mode on/off - i.e. if you have HDR on, try turning it off to see if that lets you use CrayEye without issues: https://support.google.com/camerafromgoogle/answer/10306136?hl=en

[nigelp]

Hiya @alexdredmon , one thing I've noticed is you don't have 'jpg' in your list of supported image formats. You have 'jpeg' only. Could that be a problem? The Samsung produces .jpg images.

[alexdredmon]

@nigelp hmm possibly - I'm having issues reproducing on my Android emulator but it may be particular to device/OS or camera settings.

Would you be able to provide some basic diagnostic info about your device please? e.g. Android version, device model, HDR settings

https://support.google.com/android/answer/7680439?hl=en

[nigelp]

@alexdredmon sure thing. It's a Samsung Galaxy S20 FE 5G running Android 13, OneUI 5.1. No HDR set or Scene Optimizer. Just tracking auto-focus. Attaching a new screengrab of a different error message?

I see you've now added a different model settings feature. Nice! :)

I would really like to help get your app working. :)

[alexdredmon]

@nigelp awesome, thanks for this! My emulator and device are both running Android 14 but I'll try testing with 13 to see if I get the same results you are.

So the error you're getting now is because you've selected the Ollama example engine template without updating it - in order to use a self hosted model like that you'll need to follow a few more steps (and have a machine capable of running an Ollama hosted vision model). Here's what you'll need to do for that:

1. Install Ollama on your desktop/laptop: https://github.com/alexdredmon/crayeye?tab=readme-ov-file#step-1-install-ollama
2. Expose your Ollama server via tunnel (you can skip this step if you intend to use your Ollama server's hostname to connect instead although this may limit you to only being able to use on your local network): https://github.com/alexdredmon/crayeye?tab=readme-ov-file#step-2-expose-ollama
3. Update the Ollama engine in CrayEye to use the tunnel/server hostname from step 2 (i.e. edit the engine and replace "REPLACE_WITH_YOUR_HOSTNAME" with the hostname of your server): https://github.com/alexdredmon/crayeye?tab=readme-ov-file#step-3-add-custom-engine

Hope this makes sense - let me know if you experience any issues or are able to get things working using a self hosted model and I'll report back once I'm able to test on Android 13.

[nigelp]

Ah sorry @alexdredmon , I'm not trying to run a local model. I am/was happy to run with the OpenAI API key. But it seems to have disappeared as an option?

[alexdredmon]

@nigelp yes, manually setting an API key for the default engine has been deprecated but you can achieve the same effect by creating a custom engine via settings - here's a link to add one for OpenAI: Add OpenAI Engine

Then just replace {apiKey} with your API key, save the engine, and select it.

{
  'url': 'https://api.openai.com/v1/chat/completions',
  'method': 'POST',
  'headers': {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer {apiKey}',
  },
  'body': {
    'model': 'gpt-4o',
    'messages': [
      {
        'role': 'user',
        'content': [
          {
            'type': 'text',
            'text': '{prompt}',
          },
          {
            'type': 'image_url',
            'image_url': {
              'url': '{imageUrl}',
            }
          }
        ]
      }
    ],
    'stream': true,
  },
  'responseShape': ['choices', 0, 'delta', 'content']
}`
``

[nigelp]

OK @alexdredmon progress. No error messages. :) Just a whirring circle as the app fails to upload the image using the OpenAI API. I tried to check your instructions, but that link you sent is broken. So I just added the code with the API key. Not sure what's wrong. Sorry to be THAT user!