search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
20.54k stars 1.18k forks source link

feat: support Nvidia NIMs remote API #2854

Open 0xSage opened 1 month ago

0xSage commented 1 month ago

Problem A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Request from user: https://build.nvidia.com/explore/discover so this is something I came across. With this we can get access to dbrx, mixtral8x22b through nvdia api will be good to have them

Success Criteria A clear and concise description of what you want to happen.

Additional context Add any other context or screenshots about the feature request here.

Realmbird commented 1 month ago

@0xSage Could I try this first issue I might need help doing this since I will be a beginner

Realmbird commented 1 month ago

Mixtral is https://build.nvidia.com/mistralai/mixtral-8x22b-instruct Dbrx https://docs.api.nvidia.com/nim/reference/databricks-dbrx-instruct-infer

Van-QA commented 1 month ago

hi @Realmbird, feel free to pick it up and draft a PR 🙏 We highly value the contributions of users

Realmbird commented 1 month ago

Current draft https://github.com/janhq/jan/pull/2882

Realmbird commented 1 month ago

Fixed models not holding params currently working on fixing ids with / in name

Van-QA commented 2 weeks ago

Need to correct the ‌URL for API key ‌in the settings: https://discord.com/channels/1107178041848909847/1186884274637516860/1248139642352697375 Please refer‌ to this correct flow, thank you @namchuai