search

intel / auto-round

Advanced Quantization Algorithm for LLMs/VLMs. This is official implementation of "Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs"
https://arxiv.org/abs/2309.05516
Apache License 2.0
261 stars 22 forks source link

AutoRound/Examples dependency is injected to AutoRound pkg dependency #177

Closed Qubitium closed 3 months ago

Qubitium commented 4 months ago

GPTQModel has merged AutorRound integration via PR https://github.com/ModelCloud/GPTQModel/pull/166 but we find the the CI tests is importing auto-gptq (via auto-round depends) as dependency which is strange to us since the only auto-gptq code in AutoRound is in examples.

Can examples/require*txt not be merge into auto-round pkg depends proper?

wenhuach21 commented 4 months ago

GPTQModel has merged AutorRound integration via PR ModelCloud/GPTQModel#166 but we find the the CI tests is importing auto-gptq (via auto-round depends) as dependency which is strange to us since the only auto-gptq code in AutoRound is in examples.

Can examples/require*txt not be merge into auto-round pkg depends proper?

Thank you for integrating Autoround. We should have not merge the example/requirement.txt in the main branch.

During the model packing, we need to import some packing functions from autogptq.

BTW, our API has changed a little to support the new features

Qubitium commented 4 months ago

@wenhuach21 We are closely following pending v0.3 autoround release in https://github.com/ModelCloud/GPTQModel/issues/179 as there are lots of feature and internal api change. Is there an estimated release date for autoround v0.3?

wenhuach21 commented 4 months ago

@wenhuach21 We are closely following pending v0.3 autoround release in ModelCloud/GPTQModel#179 as there are lots of feature and internal api change. Is there an estimated release date for autoround v0.3?

about 2 weeks

wenhuach21 commented 3 months ago

sorry for the late. V0.3 has been released