Fix convert script cannot generate bf16 weights

google / jetstream-pytorch

PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"

Apache License 2.0

21 stars 12 forks source link

Closed lsy323 closed 1 month ago

lsy323 commented 1 month ago

Before this PR, the config logic in convert_checkpoints.py is broken, only quantized checkpoint can be generated.

After the fix:

Use --quantize_weights to control if checkpoint will be quantized.
Use --quantize_type to specify the quantization type, only effective when --quantize_weights is true.