epic: `cortex ps` and Process Status

[dan-homebrew]

Goal

• cortex ps lists running models and free/used memory
• We should ideally be able to show VRAM and RAM use (free vs. total)

Key Questions

• How do we do this in a stateless manner? (or persist a *.lock to Cortex data folder?)

Linked PRs

• [x] https://github.com/janhq/cortex.cpp/pull/1363
• [x] https://github.com/janhq/cortex.cpp/pull/1345

DX

cortex start

• should stdout a link for api playground
• should correctly handle if port is occupied

  20240919 09:30:08.293674 UTC 3351741 INFO  Host: 127.0.0.1 Port: 3928
  - main.cc:32
  Server started

  Current logs a bit confusing

cortex stop

cortex ps

• if no active processes, should return a msg to that effect

[gabrielle-ong]

@vansangpfiev @namchuai v124, cortex ps on Windows seems broken VM: test-windows-app-1-rdp

Works well on Mac and Ubuntu

[vansangpfiev]

This is not cortex ps command issue. The root cause is the SIGINT signal (Ctrl + C) terminated the child process (server process). It works well when we use exit() to exit chat. Let me try to handle the SIGINT signal in this case.

[vansangpfiev]

@gabrielle-ong Please update cortex.llamacpp engine to latest (v0.1.34) to use the cortex ps command

cortex engines install cortex.llamacpp

[0xSage]

What attributes are we showing when there are models running?

For reference this is docker:

[gabrielle-ong]

CLI UX request: cortex ps should show model alias

• since that is expected for cortex models stop commands

Models list