-
run on Mac M3 Max 128GB
run this code
```
from transformers import AutoModel, AutoTokenizer
MAX_LENGTH = 128
model = AutoModel.from_pretrained("unsloth/Meta-Llama-3.1-405B-Instruct-bnb-4b…
-
### Feature Request
-
### System Info
Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.2
[pip3] torch==2.0.1+cu118
[pip3] torchaudio==2.0.2+cu118
[pip3] torchvision==0.15.2+cu118
…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
Hey all, thanks for your work. It seems there's an official gguf release for the 1.0 version of the Lllama-based model, but not of the 1.1 version. Is that because llama-cpp changes would be required?…
-
![image](https://github.com/user-attachments/assets/8231283e-8585-45b7-94ca-88b3a35a0771)
This config used to work on the older hugging chat 0.8.2
All my other models (OpenAI, Anthropic) work f…
-
Llama 3.1 models now officially supports function calling. So I think it would be nice to have function calling support for Llama 3.1 alongside existing functionary models.
Looks like we have mor…
-
**Title of the talk/workshop**
Building a RAG App with Groq (Llama-3), Hugging Face opensource model and Llama-Index: A Love-Hate Story
**Abstract of the talk/workshop**
How to build a RAG ap…
-
## Description
I'm deploying meta-llama/Meta-Llama-3.1-70B-Instruct on a SageMaker endpoint:
- latest DJLServing container with Neuron support (0.29)
- ml.inf2.48xlarge instance.
Model downloa…
-
### System Info
```shell
I was reading that Gaudi supports Llama 3.1 -- https://www.intel.com/content/www/us/en/developer/articles/technical/intel-ai-solutions-support-meta-llama-3-1-launch.html
A…