search

intel / neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
https://intel.github.io/neural-compressor/
Apache License 2.0
2.23k stars 257 forks source link

Integrate AutoRound v0.3 #1925

Closed Kaihui-intel closed 4 months ago

Kaihui-intel commented 4 months ago

Type of Change

feature

Description

modify:
        n_samples -> nsamples
        n_blocks -> nblocks
quant config add:
        act_bits: int = 32,
        act_group_size: int = None,
        act_sym: bool = None,
        act_dynamic: bool = True,

other config:
        multimodal:bool = False,
        use_layer_wise: bool = False, ->  low_cpu_mem_usage: bool = False, (AutoRound repo)

Expected Behavior & Potential Risk

the expected behavior that triggered by this PR

How has this PR been tested?

how to reproduce the test (including hardware information)

Dependency Change?

any library dependency introduced or removed

github-actions[bot] commented 4 months ago

⚡ Required checks status: All passing 🟢

Groups summary

🟢 Code Scan Tests workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [Code-Scan](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32833) | success | | ✅ | | [Code-Scan (Bandit Code Scan Bandit)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32833&view=logs&jobId=343c57fa-283e-589b-e772-0a0553c93e53) | success | | ✅ | | [Code-Scan (DocStyle Code Scan DocStyle)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32833&view=logs&jobId=c1e234ec-db76-5d40-e8f0-e1ad3ef905a3) | success | | ✅ | | [Code-Scan (Pylint Code Scan Pylint)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32833&view=logs&jobId=454075da-6b11-57a5-edf2-4c5947924fa8) | success | | ✅ | These checks are required after the changes to `neural_compressor/torch/algorithms/weight_only/autoround.py`, `neural_compressor/torch/quantization/algorithm_entry.py`, `neural_compressor/torch/quantization/config.py`.
🟢 Model Tests 3x workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [Model-Test-3x](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32835) | success | | ✅ | | [Model-Test-3x (Generate Report GenerateReport)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32835&view=logs&jobId=131b5a5d-c16f-50a4-e704-41ef17f1e502) | success | | ✅ | | [Model-Test-3x (Run PyTorch Model opt_125m_woq_gptq_int4)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32835&view=logs&jobId=e3333e26-4334-5d5e-a7bd-ee7471860e42) | success | | ✅ | | [Model-Test-3x (Run PyTorch Model opt_125m_woq_gptq_int4_dq_bnb)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32835&view=logs&jobId=2ce7611e-1b4a-5a32-2344-8e2c635b00bd) | success | | ✅ | | [Model-Test-3x (Run PyTorch Model opt_125m_woq_gptq_int4_dq_ggml)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32835&view=logs&jobId=919e49eb-e265-548b-eaa7-50d75ec3f230) | success | | ✅ | These checks are required after the changes to `neural_compressor/torch/algorithms/weight_only/autoround.py`, `neural_compressor/torch/quantization/algorithm_entry.py`, `neural_compressor/torch/quantization/config.py`.
🟢 Unit Tests 3x-PyTorch workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [UT-3x-Torch](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32834) | success | | ✅ | | [UT-3x-Torch (Coverage Compare CollectDatafiles)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32834&view=logs&jobId=71384379-497b-5787-1f51-cc2e0f831d78) | success | | ✅ | | [UT-3x-Torch (Unit Test 3x Torch Unit Test 3x Torch)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32834&view=logs&jobId=ce119872-54c8-5686-93fc-b763560515d2) | success | | ✅ | | [UT-3x-Torch (Unit Test 3x Torch baseline Unit Test 3x Torch baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32834&view=logs&jobId=337dfcaa-e49a-58ed-d835-0606580c9539) | success | | ✅ | These checks are required after the changes to `neural_compressor/torch/algorithms/weight_only/autoround.py`, `neural_compressor/torch/quantization/algorithm_entry.py`, `neural_compressor/torch/quantization/config.py`, `test/3x/torch/quantization/weight_only/test_autoround.py`, `test/3x/torch/requirements.txt`.

Thank you for your contribution! 💜

Note This comment is automatically generated and will be updates every 180 seconds within the next 6 hours. If you have any other questions, contact chensuyue or XuehaoSun for help.

thuang6 commented 4 months ago

It is v0.2.1 instead of v2.1