-
Hello. Can you please tell me which evolutionary search hyperparameters (population_size, mutation_numbers, crossover_size, etc.) you used to 8x increase the context length of the Mistral v0.1 or LLaM…
-
I have tried to convert llama 2 model from .gguf to .bin
```
~/llm_inferences/llama.cpp/models/meta$ ls
llama-2-7b.Q4_K_M.gguf
python3 export.py llama2_7b.bin --meta-llama /home/####/llm_inf…
-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue y…
-
Any failure in SGMV comes back as `Request failed during generation: Server error: No suitable kernel. dtype=Half`
From Discord:
> I have tried the finetune adapter for llama2-7b. I trained mode…
-
Thanks for your great work, but due to my poor knowledge, what's the 'hf_token' in your code of inference? How can I get it?
-
Hi I am trying to run sample from here -> https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/GPU/HF-Transformers-AutoModels/Model/llama2/generate.py
I am getting error as screens…
-
Hello, trying to run this in a google colab notebook and I get : **"./llama2_q4: No such file or directory"**
when running : **!./llama2_q4 llama2-7b-awq-q4.bin -n 256 -i "write an essay about GPUs"*…
-
Hello,
I tried to reproduce the results of the paper, and got similar results for Llama2-7B, 13B, 70B, and Llama-3 8B.
However, when I tested Llama3-70B using the optimized rotation matrix you p…
-
Hi, I appreciate your work! I have a question regarding the zero-shot common sense reasoning task on llama2-7. I tested llama2-7b 4-4-4 using lm-eval and observed a significant discrepancy in the resu…
-
Traceback (most recent call last):
File "/home/m00830934/code/LongRoPE/evolution/evaluate.py", line 110, in
main(args)
File "/home/m00830934/code/LongRoPE/evolution/evaluate.py", line 52, …