search

modal-labs / modal-examples

Examples of programs built using Modal
https://modal.com/docs
MIT License
703 stars 160 forks source link

how to run llama3.2 1b model ? #900

Closed Arslan-Mehmood1 closed 1 day ago

Arslan-Mehmood1 commented 1 day ago

using this

06_gpu_and_ml/llm-serving/text_generation_inference.py

I'm unable to get generated text correctly. image

charlesfrye commented 1 day ago

Hey there!

We reserve the GitHub Issues page for issues with the examples as they are written, but this looks like an issue with the example after it has been edited.

We provide support for general use of the Modal platform in our Community Slack, https://modal.com/slack.

The short answer is that it looks like the chat template here is wrong.