-
When I run the command:
python -m fastchat.model.apply_delta --base {path_to_llama_weights} --target ./vicuna_ckpt/7b_v0/ --delta {path_to_delta_vicuna_weights}
I got this error:
Unable to…
-
**The following are based on question \ answer of 1 document with 22769 tokens length**
there is a similar issue https://github.com/imartinez/privateGPT/issues/276 with primordial tag, just decide…
-
### System Info
transformers==4.42.3
### Who can help?
@zucchini-nlp
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An official…
-
How to fine tune vicuna-7b with A40
-
You fine-tuned the language model, etc., using lora fine-tuning on the basis of the original model llava-hf/llava-v1.6-vicuna-7b-hf, but your open source weights (ermu2001/pllava-7b) seem to contain o…
-
Running inference using vicuna 7B on a 16Gb 3080. Occasionally the script crashes with an error like:
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 16.00 GiB total capacity; 1…
-
### What happened?
LLaMA/Vicuna 7B fp16 model is producing all zero results on CPU for both local-sync and local-task.
### Steps to reproduce your issue
1. Download the .mlir from https://sto…
-
Hi,
Willing to use the `Vicuna v1.5 7b` model for RAG or Retrieval Augmented Generation (Q&A based on retrieved documents or context), I tried many prompts based on Llama 2 prompting but never mana…
-
How can we use vicuna as the LLM in inference code. Below is the error message. Thanks.
-
First, thanks very much for creating this cool technology.
On one A100 GPU w/ 80GB VRAM, I tried benchmarking `sq-vicuna-7b-v1.3-w3-s0 ` and its base. It is a bit strange that running median time h…