[Bug]: OpenRouter support seems broken

[czottmann]

Before Reporting

• [X] I have checked FAQ, and there is no solution to my issue
• [X] I have searched the existing issues, and there is no existing issue for my issue

What happened?

I've configured some OpenRouter models, and they worked well. About two days ago, Copilot Chat started erroring out ("The data couldn’t be read because it isn’t in the correct format."). I've checked the chat model setup, and noticed that clicking "Test" will return an error ("The data couldn't be read because it is missing.")

I've made sure the OpenRouter API key is correct and works by testing an API call via curl.

Additionally, I can use OpenAI just fine. When I configure the Chat feature to use OpenAI across the board (see screenshot 1 below), I get responses in the chat.

Changing it to use an OpenRouter model (see screenshot 2 below), the chat only returns "The data couldn’t be read because it isn’t in the correct format.". It seems to work intermittently when I set the preferred chat model for Sense/Project scope to "OpenAI". Disabling "Enable by default" doesn't make a difference. Using another OpenRouter model yields the same results.

Please let me know if I can provide more info, or if there's anything else I can do. Thanks!

How to reproduce the bug.

It just happened!

Relevant log output

No response

macOS version

14.3.1

Xcode version

15.2

Copilot for Xcode version

0.31.0

[intitni]

The new version has migrated to use the tool calling API instead of the function calling API. I have no clue but my guess is that OpenRouter is still using the function calling API. Please try turning off function calling support for the model.

Don't worry, function/tool call is only used in a few unimportant features. And they seems to only work for OpenAI models.

[intitni]

I gave it a try just now, I was wrong.

[intitni]

Fixed in 0.31.1

[czottmann]

Wow, super-fast turnaround! You're a legend, many thanks 🤙🏼

[czottmann]

@intitni While it's working now for small questions, as soon as I enter more involved tasks (like "rewrite this curl command as a Swift Alamofire call"), Copilot will crash.

[intitni]

@czottmann Please send me the crash log. You can find it in Console.app.

[czottmann]

Four crashlogs coming up: CopilotForXcodeExtensionService-2024-03-13-ips.zip

[intitni]

@czottmann I don't understand the crash. It was trying to access a global let, then it crashed..

Please let me know if this version fixes it.

https://github.com/intitni/CopilotForXcode/releases/download/0.1.0/Copilot.for.Xcode.app.debug.20240314.zip

[czottmann]

Doesn't fix it, unfortunately. But I can reliably reproduce the issue with the following request, maybe that's of help:

Rewrite the following curl call into a Swift AlamoFire function:

curl \
  --request POST https://api.keygen.sh/v1/accounts/e9087ff8-fd4a-4595-8179-1581de1708ea/licenses/actions/validate-key \
  --header 'Content-Type: application/vnd.api+json' \
  --header 'Accept: application/vnd.api+json' \
  --data '{
    "meta": {
      "key": "50db2a31-b769-4e71-8f8f-f943a0b1eee2",
      "scope": {
        "product": "aef5f60d-f080-4445-a34b-d7bfb620275a"
      }
    }
  }' 

The chat will either stall and beachball or crash after a while. I can see OpenRouter sending back responses (list at https://openrouter.ai/activity), though.

[intitni]

Please upload the new crash log. If you see beach ball, please try exporting the spin dump from Activity Monitor.app.

Also let me know if turning off project scope helps.

[intitni]

I think the reason is that a number was either infinite or nan during calculation and caused a Swift runtime error. Please give this version a try:

https://github.com/intitni/CopilotForXcode/releases/download/0.1.0/Copilot.for.Xcode.app.debug.20240314.2.zip

[czottmann]

This latest build did the trick, thank you! It's working as expected again.