-
### 🚀 The feature, motivation and pitch
Currently, there is a [parallel_tool_calls](https://github.com/vllm-project/vllm/blob/18b296fdb2248e8a65bf005e7193ebd523b875b6/vllm/entrypoints/openai/protocol…
-
I submitted a request for access and obtained a key from the following URL: [https://llama.meta.com/llama-downloads/](https://llama.meta.com/llama-downloads/)
Instructions refer to download refer t…
-
I tried these two quantization approaches:
```
model_path = '/home/catid/models/Meta-Llama-3-70B-Instruct'
quant_path = 'cat-llama-3-70b-q128-w4-gemvfast'
quant_config = { "zero_point": True, "q…
catid updated
6 months ago
-
hi,
For model meta-llama/Meta-Llama-3-8B-Instruct, it is unlisted, not sure when will be supported?
https://github.com/huggingface/chat-ui/blob/3d83131e5d03e8942f9978bf595a7caca5e2b3cd/.env.templa…
cszhz updated
6 months ago
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch…
-
### System Info
WARNING:root:A model is loaded from './saved_models/fp-meta-llama3', and no v_head weight is found.
### Information
- [ ] The official example scripts
- [X] My own modified scripts…
-
When i try to reproduce the result following the instruction in READEME, I get the following result in TruthfulQA for Llama-2-7b. AUROC is **60.36**, which is far from **78.64** in Table 1. The full o…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
I have the code below and it runs well. However when I inspect the 'response' variable t…
-
Hello everyone, I have a problem and would like to ask for help. After I compile and run the inference code run.py, if I set max_output_len to a small value, the output will be truncated before it is …
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
Latest LLaMA-Factory repo 12Septr2024 forces to use Torch 2.4 hence is clashing with Unsloth/XFormers
##…