-
I have searched in “[Model Cards and Prompt Formats](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/)” and in the Github repositories about what is the correct way to reference a …
-
April 2024.
-
I've been trying to use unsloth with the following code:
```
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth…
-
Hope you can help with this. I'm trying to implement ring attention using Llama 3 architecture and I'm starting with the blockwise parallel transformer piece. My question is when do I start to break t…
-
Right now, when we finetune a LoRA on top of e.g. Llama 3.1 8B instruct, even if model_name is `meta-llama/Meta-Llama-3.1-8B-Instruct`, it gets resolved to `unsloth/meta-llama-3.1-8b-instruct-bnb-4bit…
-
Thanks for adding tool/with_structured_output support to ChatBedrockConverse!
It is working great for models like mistral_large. However, the Llama 3.1 models don't work with with_structured_output…
-
### Proposal to improve performance
_No response_
### Report of performance regression
I have a single machine 8xH100 SXM system. I wanted to see how vLLM (0.5.5) compares to other engines with TP=…
-
Opening a new issue for the previously opened issue here -- https://github.com/huggingface/tokenizers/issues/1517
Here we can see that the desired behavior for `return_offsets_mapping` from Mistral…
-
Llama 3.1 got released, anything needs to change to get it working.
jkfnc updated
1 month ago
-
问一下博主,支持加速Meta-Llama-3-70B-Instruct吗,我用您的方法加速Meta-Llama-3-70B-Instruct结果回复的都是反斜杠
![image](https://github.com/ztxz16/fastllm/assets/173896419/eb696cab-aae0-4295-bca5-9a34646bb8bf)