search

google / jetstream-pytorch

PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"
Apache License 2.0
32 stars 14 forks source link

Use GemmaAttention for Gemma #72

Closed qihqi closed 3 months ago

qihqi commented 3 months ago

This way it produces more accurate results (with EOS)

{'rouge1': 36.9881, 'rouge2': 13.3464, 'rougeL': 21.7437, 'rougeLsum': 35.1489, 'gen_len': 1295948, 'gen_num': 1000}