clarify that no quantization config needs to be specified for fp8 405B

huggingface / huggingface-llama-recipes

531 stars 59 forks source link

Closed ianporada closed 1 month ago

ianporada commented 1 month ago

fix typo
add clarification regarding quantization config
torch_dtype is already specified in the model config.json so we can just use "auto"

ianporada commented 1 month ago

@SunMarc fixed merge conflicts

SunMarc commented 1 month ago

Nice thanks again for the contribution !