search

janhq / cortex.cpp

Local AI API Platform
https://cortex.so
Apache License 2.0
2.04k stars 115 forks source link

hardware: Windows on ARM support (with Snapdragon X series NPU) #928

Open high3eam opened 3 months ago

high3eam commented 3 months ago

Is your feature request related to a problem? Please describe it

I was wondering if jan supports Windows on ARM platform, specifically on Snapdragon X series chips. Is it already planned to support this platform and potentially allow the use of the integrated NPU? It looks like closed source competitor "LMStudio" supports the platform already (BETA, and still without NPU support).

Describe the solution

Support for Windows on ARM and integrated NPU of Snapdragon X series chips.

Teachability, documentation, adoption, migration strategy

No response

What is the motivation / use case for changing the behavior?

Since Windows on ARM laptop are taking of with the newly released qualcomm ARM chips, it would be great if Jan natively supported it.

0xSage commented 1 month ago

We're working on supporting ONNX format models through DirectML, which natively accelerates on NPUs. There are some models available already, but consider this a work in progress https://cortex.so/models/