search

janhq / cortex.cpp

Run and customize Local LLMs.
https://cortex.so
Apache License 2.0
1.96k stars 111 forks source link

bug: Failed to read connection when running model #1447

Open JulienSorin opened 1 week ago

JulienSorin commented 1 week ago

Cortex version

cortex-1.0.0-linux-amd64-local-installer.deb

Describe the Bug

Impossible to run model in docker.

Steps to Reproduce

  1. docker run -it debian:trixie /bin/bash
  2. apt update && apt install -y wget sudo
  3. cd ~ && wget -O cortex.deb https://github.com/janhq/cortex.cpp/releases/download/v1.0.0/cortex-1.0.0-linux-amd64-local-installer.deb
  4. apt install ./cortex.deb
  5. cortex engines list
  6. cortex pull llama3.1:8b-gguf
  7. cortex run llama3.1:8b-gguf

Screenshots / Logs

{7FDDFC03-8578-48F7-954D-6A89E23CB00C}

...

{C49A2928-3828-40E9-9543-58CE28EAABC7}

{D08B2726-AFDD-441F-8F60-F8AC9515DE6C}

{1142C160-1942-451E-A226-FA2C538FB395}

{4360B2F5-53F3-49E7-90CE-AF4EB1A98FEF}

What is your OS?

What engine are you running?

vansangpfiev commented 1 day ago

Hi @JulienSorin, can you please share the cortex logs? They are in /root/cortexcpp/logs folder