search

NexaAI / nexa-sdk

Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.
https://docs.nexa.ai/
Apache License 2.0
4.42k stars 653 forks source link

[QUESTION] <title>nexa server response `Aborted (core dumped)` #257

Open SunJiaheng66 opened 1 week ago

SunJiaheng66 commented 1 week ago

Question or Issue

When i download model form huggingface and Official website for the OmniVision. I try to run the nexa server on linux, but it always response Aborted (core dumped) image image When i used huggingface model, it also responses this error. How can i fixed this error.

OS

Ubuntu

Python Version

3.10.12

Nexa SDK Version

0.0.9.1

GPU (if using one)

NVIDIA A800

Davidqian123 commented 1 week ago

For now, we don't support nexa server omnivision temporarily, you can make a feature request for it, and we will implement soon!

SunJiaheng66 commented 1 week ago

How can i use omnivision to lager requset or large inference. I also occured this problem.

model_path = "/opt/cv/models/cv/Nexa-OmniVision/"
inference = NexaVLMInference(
    model_path=model_path ,
    local_path=None,
    stop_words=[],
    temperature=0.7,
    max_new_tokens=2048,
    top_k=50,
    top_p=1.0,
    profiling=True,
    debug=True
)

It always response /opt/cv/models/cv/Nexa-OmniVision/ does not have 2024-11-21 03:05:28,964 - ERROR - Model or projector not found in .. Make sure to name them as 'model-<tag>.gguf' and 'projector-<tag>.gguf'. but i have downloaded this models.

SunJiaheng66 commented 1 week ago

image This is my code. image This is the models which i downloaded form Official website. But it always responses this error. image Please help me resolvse this error.

Davidqian123 commented 1 week ago

For now, we only implement CLI for omnivision, python interface and server are on the road map. We also welcome community to contribute to nexa-sdk to help us export NexaOmniVlmInference python interface and server.

SunJiaheng66 commented 2 days ago

img_v3_02h0_d6de9213-e830-4f20-ad9f-1d1b746f90lu When i use the nexa sdk installed on windows, i want to test the proformance for the Nexa-OmniVision, but it always response the error for the Chinese questions as the input. Does it do not support the Chinese input? I used the nexa run --local_path XXX --model_type MULTIMODAL to start the client. Please help me resolvse this error.

SunJiaheng66 commented 2 days ago

img_v3_02h0_ac4e28d4-3401-45f8-a2bc-54484dbee7lu Annother question i want to confirm, my window cuda only supports the CUDA11.7, i installed the cpu SDK and CUDA SDK, but when i use CUDA SDK, the CUDK memory does not increase. I want to confirm the CUDA SDK is not support or the using method is error. I used the nexa run --local_path XXX --model_type MULTIMODAL to start the client.