Sync model - Githubissues

deep-diver / llamaduo

This project showcases an LLMOps pipeline that fine-tunes a small-size LLM model to prepare for the outage of the service LLM.

https://huggingface.co/papers/2408.13467

Apache License 2.0

286 stars 29 forks source link

Closed deep-diver closed 5 months ago

deep-diver commented 5 months ago

alignment's get_quantization_config to get quantization config matching to the ones used in fine-tuning
reused the practices from alignment to set torch_dtype and model_kwargs

deep-diver commented 5 months ago

@sayakpaul more thoughts?

sayakpaul commented 5 months ago

No all good.