search

janhq / cortex.cpp

Local AI API Platform
https://cortex.so
Apache License 2.0
2.14k stars 127 forks source link

ci: CI packaging of llama.cpp dependencies into the binary file by default for Cortex's integration into Jan #1369

Open hiento09 opened 1 month ago

hiento09 commented 1 month ago

Linked to #1217 Given that: The current CI is publishing the binary file and installer to the release, but the binary does not include the llamacpp engine. Expectation: Jan will pull the Cortex binary that has the llamacpp engine avx2 pre-integrated.

Bugs:

hiento09 commented 1 month ago

Approach:

  1. Update Cortex CI: Similar to the old approach like Nitro, ship multiple Cortex binaries including different llamacpp variants. Image
  2. Update Jan CI: Modify Jan's CI to pull llamacpp engines from the cortex.llamacpp repo and the Cortex binary file from the cortex repo.
hiento09 commented 1 month ago

cc @dan-homebrew @louis-jan @vansangpfiev

gabrielle-ong commented 1 month ago

@hiento09 is this a Sprint21 issue and is there a corresponding PR? Else we can shift it to Sprint22 cc @dan-homebrew

hiento09 commented 1 month ago

@hiento09 is this a Sprint21 issue and is there a corresponding PR? Else we can shift it to Sprint22 cc @dan-homebrew

Let shift this issue to sprint 22 @gabrielle-ong , we have not finalized the approach yet