search

janhq / cortex.cpp

Local AI API Platform
https://cortex.so
Apache License 2.0
2.1k stars 122 forks source link

bug: Failed to read connection when running model #1447

Open JulienSorin opened 1 month ago

JulienSorin commented 1 month ago

Cortex version

cortex-1.0.0-linux-amd64-local-installer.deb

Describe the Bug

Impossible to run model in docker.

Steps to Reproduce

  1. docker run -it debian:trixie /bin/bash
  2. apt update && apt install -y wget sudo
  3. cd ~ && wget -O cortex.deb https://github.com/janhq/cortex.cpp/releases/download/v1.0.0/cortex-1.0.0-linux-amd64-local-installer.deb
  4. apt install ./cortex.deb
  5. cortex engines list
  6. cortex pull llama3.1:8b-gguf
  7. cortex run llama3.1:8b-gguf

Screenshots / Logs

{7FDDFC03-8578-48F7-954D-6A89E23CB00C}

...

{C49A2928-3828-40E9-9543-58CE28EAABC7}

{D08B2726-AFDD-441F-8F60-F8AC9515DE6C}

{1142C160-1942-451E-A226-FA2C538FB395}

{4360B2F5-53F3-49E7-90CE-AF4EB1A98FEF}

What is your OS?

What engine are you running?

vansangpfiev commented 1 month ago

Hi @JulienSorin, can you please share the cortex logs? They are in /root/cortexcpp/logs folder

gabrielle-ong commented 1 month ago

@JulienSorin following up - can you share your cortex logs, as well the Linux version you are using?

gabrielle-ong commented 3 weeks ago

Hi @JulienSorin, we've released cortex v1.0.1 and fixed lots of bugs We'll love if you can give it another go at running llama3.1.

To update to cortex v1.0.1 (or download here: https://cortex.so/)

> cortex update
> cortex update --server