janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
22.26k stars 1.28k forks source link

bug: O1-Preview does not work because of: "Unsupported parameter: 'max_tokens' is not supported with this model.". #3698

Closed raphaelmosaic closed 20 hours ago

raphaelmosaic commented 6 days ago

Jan version

v0.5.4-640

Describe the Bug

When making a request using o1-preview or o1-mini, getting:

Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead. Jan’s in beta. Access troubleshooting assistance now.

Cannot change these model parameters settings.

Steps to Reproduce

No response

Screenshots / Logs

c9e27463a830030ab08bc96bb4282cd72091d9cb9a9b4f320d722585b368bb33

What is your OS?

imtuyethan commented 1 day ago

Thanks for reporting this @raphaelmosaic.

https://community.openai.com/t/why-was-max-tokens-changed-to-max-completion-tokens/938077/3

Looks like the new o1 series models (like o1-preview and o1-mini) use max_completion_tokens instead of max_tokens. We'll work on updating Jan to handle this smoothly.

For now, here's a quick workaround:

  1. Navigate to your Jan data folder (usually in your user directory)
  2. Find models folder, then find the models/o1-mini/model.json file (and o1-preview if you use that too)
  3. Open it in a text editor
  4. Change max_tokens to max_completion_tokens
  5. Save the file and restart Jan
Screenshot 2024-09-23 at 2 55 56 PM

This should get you up and running. We'll be pushing an update soon to handle this automatically.

raphaelmosaic commented 1 day ago

Changed this, still does not work. Please fix this properly asap as I am 100% one of many people who would use o1mini / o1preview via Jan and have this issue

CleanShot 2024-09-23 at 22 09 30 CleanShot 2024-09-23 at 22 09 37

louis-jan commented 22 hours ago

Build containing bug fixes: https://github.com/janhq/jan/actions/runs/11007851785