-
Given that we have only Llama 3 70B and 8B, it would be useful to have a Tiny Llama based on the Llama 3 tokenizer so that we can use it as a drafting model for speculative decoding.
Are there pla…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
Do you have plans to support other LLM models like Llama 3?
Or would it be easy to modify code implementing interface to OpenAI. I would like inerface using Ollama.
Any hints would be appreciate…
-
when i run "torchrun --nproc_per_node 1 /opt/Meta-Llama-3-8B/example_text_completion.py --ckpt_dir /opt/Meta-Llama-3-8B/ --tokenizer_path /opt/Meta-Llama-3-8B/tokenizer.model"
i got an error …
-
In both the example_text and example_chat the package 'llama' is required. I cannot install a compatible package 'llama'. The one I find was last updated 2017 and is only Python2.7 compatible! Any poi…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### Reproduction
Hi There, I am observing a difference in output between llama factory inference and llama.cpp.
I am…
anidh updated
3 weeks ago
-
Track issues which will improve Llama e2e perf
- [ ] (Colman) get all `set_runtime_args` calls below 3 microseconds
## Runtime-specific
`wait_for_tensor_data_populated` in worker threads has …
-
Hey
I've trying to use llmstudio cli since I do not have enough resources required by the H2o llmstudio. So I use Kaggle to run my cli tool.
But anyways, I'm trying to train Llama-2-7b on my own da…
-
Is LLaMA-7B LLM the original model or Llama-2-Chat model optimized for dialogue use cases?
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [ X] I am running the latest code. Development is very rapid so there are no tagged versions as o…