AutoRound/Examples dependency is injected to AutoRound pkg dependency

intel / auto-round

Advanced Quantization Algorithm for LLMs. This is official implementation of "Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs"

https://arxiv.org/abs/2309.05516

Apache License 2.0

132 stars 18 forks source link

AutoRound/Examples dependency is injected to AutoRound pkg dependency #177

Open Qubitium opened 1 week ago

Qubitium commented 1 week ago

GPTQModel has merged AutorRound integration via PR https://github.com/ModelCloud/GPTQModel/pull/166 but we find the the CI tests is importing auto-gptq (via auto-round depends) as dependency which is strange to us since the only auto-gptq code in AutoRound is in examples.

Can examples/require*txt not be merge into auto-round pkg depends proper?

wenhuach21 commented 1 week ago

GPTQModel has merged AutorRound integration via PR ModelCloud/GPTQModel#166 but we find the the CI tests is importing auto-gptq (via auto-round depends) as dependency which is strange to us since the only auto-gptq code in AutoRound is in examples.

Can examples/require*txt not be merge into auto-round pkg depends proper?

Thank you for integrating Autoround. We should have not merge the example/requirement.txt in the main branch.

During the model packing, we need to import some packing functions from autogptq.

BTW, our API has changed a little to support the new features

Qubitium commented 1 week ago

@wenhuach21 We are closely following pending v0.3 autoround release in https://github.com/ModelCloud/GPTQModel/issues/179 as there are lots of feature and internal api change. Is there an estimated release date for autoround v0.3?

wenhuach21 commented 1 week ago

@wenhuach21 We are closely following pending v0.3 autoround release in ModelCloud/GPTQModel#179 as there are lots of feature and internal api change. Is there an estimated release date for autoround v0.3?

about 2 weeks