search

NexaAI / nexa-sdk

Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.
https://docs.nexa.ai/
Apache License 2.0
4.32k stars 638 forks source link

[BUG] Model Not Start after complete download on windows #115

Open Alias4D opened 2 months ago

Alias4D commented 2 months ago

Issue Description

Hi i install nexa sdk for CPU on windows 10 then run below command

nexa server Octo-planner:q8_0

but no think happen no server start ?

INFO: Started server process [6796] INFO: Waiting for application startup. 2024-09-23 22:00:11,847 - INFO - Model Path: Octo-planner:q8_0 Model Octo-planner:q8_0 already exists at C:\Users\xxx.cache\nexa\hub\official\Octo-planner\q8_0.gguf

same think for command nexa run Octo-planner:q8_0

Steps to Reproduce

i install nexa sdk for CPU on windows 10 then run below command

nexa server Octo-planner:q8_0

but no think happen no server start ?

INFO: Started server process [6796] INFO: Waiting for application startup. 2024-09-23 22:00:11,847 - INFO - Model Path: Octo-planner:q8_0 Model Octo-planner:q8_0 already exists at C:\Users\xxx.cache\nexa\hub\official\Octo-planner\q8_0.gguf

same think for command nexa run Octo-planner:q8_0

OS

Windows 10

Python Version

Python 3.11.4

Nexa SDK Version

0.0.8.4

GPU (if using one)

No response

zhiyuan8 commented 2 months ago

@Alias4D Could you please record a short screen record or give us more terminal print details?

Alias4D commented 2 months ago

OK, after I return from my travel 😊 I will provide short video 👍

Alias4D commented 2 months ago

Hi , welcom againe my screen shot Screenshot 2024-09-28 233441

for long time no think happens , no app start or any text printed on terminal

JGalego commented 1 month ago

Worked like a charm using version 0.0.8.6 image

Alias4D commented 1 month ago

I will try update

Alias4D commented 1 month ago

Welcome 👋 , again I update nexaai to last version 0.0.8.6 and try again But same state no think happen no server start .....😔

Screenshot 2024-10-03 164630

Alias4D commented 1 month ago

Welcome 👋 , again I update nexaai to last version 0.0.8.6 and try again But same state no think happen no server start .....😔

Screenshot 2024-10-03 164630

Davidqian123 commented 1 month ago

If port 9000 is occupied? Try to use nexa server Llama3.2-3B-Instruct:q4_0 or change another model to start the server.