search

foundation-model-stack / fms-acceleration

🚀 Collection of libraries used with fms-hf-tuning to accelerate fine-tuning and training of large models.
Apache License 2.0
0 stars 4 forks source link

Extract out AutoGPTQ dependency #38

Open fabianlim opened 1 week ago

fabianlim commented 1 week ago

The AutoGPTQ repo looks like it hasnt had any releases since Mar 1, 2024. While it is still a dependency of major frameworks such as huggingface and vllm, it is concerning that it might get deprecated. Maybe we can consider to extract out the relevant code since it is on an MIT license

raghukiran1224 commented 1 week ago

My suggestion is to do what inference team is doing. We can switch when they do. Thoughts?

fabianlim commented 1 week ago

@raghukiran1224 thanks for the suggestion. let me go find out more.