Add gemma support in better cli

google / jetstream-pytorch

PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"

Apache License 2.0

33 stars 14 forks source link

Closed qihqi closed 2 weeks ago

qihqi commented 2 weeks ago

python -m jetstream_pt.cli interactive --model_id meta-llama/Meta-Llama-3-8B-Instruct --quantize_weights=0