-
So, I tried to make this work, but it just doesnt load checkpoints or models in GGUF format. Also tried LLM party, just to make sure, but their loader seem to be exactly the same as this one. And it d…
-
Hello,
I have demand that deploy some vlm nano llm not supported to Orin,such as qwenvl, Can I know hwo to join other vlm to nanollm.
Many many thanks!
-
Hi,
I tried to insert values in struct to be create by asn1c.
But, i do not know how to insert Extension values, Encoding/Decoding and Accessing members.
it is code that is created by asn1c.
…
-
Hi, I’m encountering a similar CUDA out-of-memory (OOM) issue as described in #10 while running on TITAN XP GPUs to run the pascal voc experiment with 92 labels. It occurred during the ASPP stage. To …
-
Hi, thanks for your work again!
I am trying to run the experiment on one 32G GPU, but i noticed here in your previous issue #11 that the peak usage is 35G. But actually the model i am using is jus…
-
### System Info
In current implementation of VLMs, the "_supports_sdpa" attribute checks and activates SDPA attention only for the language model. For example in [Llava](https://github.com/huggingf…
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue y…
-
### Anything you want to discuss about vllm.
Can the current benchmark_serving.py be used with Multimodal LLM (llava) and image input? The existing code send the request in the following format in ba…
-
Hi,
I was trying to collect a .r3d of my front door, and I followed the ins…
-
### Describe the issue
Issue:
Failed to run the model with an error: AttributeError: 'NoneType' object has no attribute 'is_loaded'
I apologize, I am new to this so if there is a simple solution …