bug: App shell conflicts between `cortex-nightly` and `cortex-stable`

[gabrielle-ong]

Cortex version

v172

Describe the Bug

App shell conflicts between cortex-nightly and cortex-stable:

when I load a model on one version, the model is loaded on both cortex and cortex-nightly when I unload the model on one version, the model is unloaded on both cortex and cortex-nightly

Not sure if this is a cortex ps issue or installation/data folder conflict with cortex and cortex-nightly

Steps to Reproduce

No response

Screenshots / Logs

What is your OS?

• [X] MacOS
• [ ] Windows
• [ ] Linux

What engine are you running?

• [X] cortex.llamacpp (default)
• [ ] cortex.tensorrt-llm (Nvidia GPUs)
• [ ] cortex.onnx (NPUs, DirectML)

[vansangpfiev]

Our CLI and Server communicate by TCP IP socket for now and the Server does not know about cortex variant, so it is expected. This will be solved after we support Unix domain socket cc: @namchuai @nguyenhoangthuan99

[gabrielle-ong]

@vansangpfiev do we have plans to move to UDS, and should this be done earlier rather than later?

cc @dan-homebrew @0xSage

[gabrielle-ong]

Decision:

• move Bug to icebox (Sprints: 2 sprints away)
• create new Epic on moving to Unix Domain Sockets (Size: Large, Status: Planning)

[0xSage]

Dumb/nonurgent question: If we switch to UDS, what about devs directly embedding Cortex in their apps and only using the API? Will we still support TCP/IP (the most common protocol)?

[vansangpfiev]

IMO, we will support both UDS and TCP/IP. Clients (CLI, Jan, ...) can switch to which protocol they want.

[vansangpfiev]

@gabrielle-ong A temporary solution for TCP/IP is to use different ports by configuring apiServerPort in .cortexrc*.

[gabrielle-ong]

Thanks @vansangpfiev, Ill use that workaround iceboxing this bug and creating a planning ticket #1517