feat: Jan supports Anthropic's Claude Sonnet 3.5

eckartal commented 1 week ago

[ ] I have searched the existing issues

Is your feature request related to a problem? Please describe it

Jan supports Anthropic's new AI model, Claude Sonnet 3.5

Describe the solution

https://x.com/AnthropicAI/status/1803790676988920098

Teachability, documentation, adoption, migration strategy

No response

What is the motivation / use case for changing the behavior?

More models.

danwritecode commented 1 week ago

guys there should be a simple way for power users to just enter the model name when there are new model releases. it's a shame not being able to use your product because i'm waiting on a pull request to be made when i could just enter a model name and be up and running.

i get you want to vet and test new models, but if someone wants to override the default models, that's on them.

DenisGorbachev commented 6 days ago

@hahuyhoang411 Is it possible to release the version with Sonnet 3.5 please? It's more powerful than GPT-4o.

eggsyntax commented 4 days ago

For whoever ends up working on this: I tried doing the naive thing, adding the following to the Anthropic models.json file (locally) and rebuilding with make dev:

  {
      "sources": [
          {
              "url": "https://www.anthropic.com/"
          }
      ],
      "id": "claude-3-5-sonnet-20240620",
      "object": "model",
      "name": "Claude 3.5 Sonnet",
      "version": "1.0",
      "description": "Claude 3.5 Sonnet is the June 2024 version of Anthropic's mid-sized model.",
      "format": "api",
      "settings": {},
      "parameters": {
          "max_tokens": 4096,
          "temperature": 0.7,
          "stream": false
      },
      "metadata": {
          "author": "Anthropic",
          "tags": [
              "General",
              "Big Context Length"
          ]
      },
      "engine": "anthropic"
    },

but 3.5-Sonnet still doesn't show up as an available model in the app. I'm guessing there's a remote list being checked, maybe on HF? Or an additional build step other than make dev?

"id" taken from this anthropic doc page.

I haven't taken time to dig into the overall app, just had a bit of time to spare to try the simple thing, sharing in case it's of use.

joshuabrandes commented 4 days ago

I did the exact same thing. Doesn't work for me as well @eggsyntax

robertfall commented 2 days ago

i get you want to vet and test new models, but if someone wants to override the default models, that's on them.

Right now I'm looking at switching away from LibreChat to a different tool and not being able to choose my own models makes Jan a non-starter for me.

Overall it looks like a great app, but this seems like an obvious feature.

corwin1949 commented 1 day ago

Indeed, we need to be able to use models by filling in strings, otherwise, every time there is a big model upgrade in the future jan won't be able to support it in the first place.

It would be nice to be able to automatically get which models are available on the port.

0xSage commented 5 hours ago

@louis-jan to think throug hhow to handle

janhq / jan