-
**Is your feature request related to a problem? Please describe.**
Would very much like Llama 3.1 8B (or ideally 70B) support natively.
![CleanShot 2024-07-26 at 05 44 30@2x](https://github.com/us…
-
What are the memory footprints (GB) of
- Llama-3.1-8B
- Llama-3.1-70B
- Llama-3.1-405B
- Llama-3-8B
- Llama-3-70B
models and hardware specifications required to run the models?
-
I'm trying to build `Llama 3.1` and `Llama 3.1 Instruct` but the build always fails (latest main or v1.2.0). These models are not supported yet?
`Llama 3` and `Llama 3 Instruct` also don't work.
…
-
Given that we have only Llama 3 70B and 8B, it would be useful to have a Tiny Llama based on the Llama 3 tokenizer so that we can use it as a drafting model for speculative decoding.
Are there pla…
-
Hey, I was trying to call Llama 3.1 on the proxy, but I got a "model not supported" error. I was looking at the code and noticed that the entry for Llama 3.1 hasn't been added yet.
https://github.…
-
Are there instructions specific to creating a bmodel from onnx for Llama 3.1 (not lllam3)
Running this is erroring out.
python export_onnx.py --model_path ../../../../Meta-Llama-3.1-8B-Instruct/ -…
-
When I run llama 3.1 example with runpd I'm getting this error:
h/sky-key' root@69.30.85.136 -p 22035 -o StrictHostKeyChecking=no -o PasswordAuthentication
=no -o ConnectTimeout=10s -o UserKn…
-
I have been staging some updates testing the tgi-gaudi software with llama 405B fp8, i am waiting for habana optimum to approve the PR, and then I will submit a pr for huggingface/tgi_gaudi and will s…
-
### Checklist
- [x] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue y…
-
Hi,
I tried training llama 3.1 with run_mntp.py but get an obsucre error
`AttributeError: 'LlamaBiModel' object has no attribute 'rotary_emb'`
What is that about ?