search

intel / neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime
https://intel.github.io/neural-compressor/
Apache License 2.0
2.18k stars 252 forks source link

fix bf16 symbolic_trace bug #1892

Closed xin3he closed 2 months ago

xin3he commented 3 months ago

Type of Change

bug fix

Description

fix bf16 symbolic_trace bug,

  1. cause abnormal recursive calling.
  2. missing necessary attributes

By moving BF16 fallback ahead of quantization and removing bf16_symbolic_trace, we fix it.

Expected Behavior

google/vit-base-patch16-224 autotuned with bf16 ops works.

How has this PR been tested?

local test

github-actions[bot] commented 3 months ago

⚡ Required checks status: All passing 🟢

Groups summary

🟢 Code Scan Tests workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [Code-Scan](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32233) | success | | ✅ | | [Code-Scan (Bandit Code Scan Bandit)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32233&view=logs&jobId=343c57fa-283e-589b-e772-0a0553c93e53) | success | | ✅ | | [Code-Scan (DocStyle Code Scan DocStyle)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32233&view=logs&jobId=c1e234ec-db76-5d40-e8f0-e1ad3ef905a3) | success | | ✅ | | [Code-Scan (Pylint Code Scan Pylint)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32233&view=logs&jobId=454075da-6b11-57a5-edf2-4c5947924fa8) | success | | ✅ | These checks are required after the changes to `neural_compressor/adaptor/pytorch.py`, `neural_compressor/adaptor/pytorch_cpu.yaml`, `neural_compressor/adaptor/torch_utils/bf16_convert.py`.
🟢 Model Tests workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [Model-Test](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32235) | success | | ✅ | | [Model-Test (Generate Report GenerateReport)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32235&view=logs&jobId=131b5a5d-c16f-50a4-e704-41ef17f1e502) | success | | ✅ | | [Model-Test (Run ONNX Model resnet50-v1-12)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32235&view=logs&jobId=8036be5b-c44f-5dbf-ab25-6a62f8567a7d) | success | | ✅ | | [Model-Test (Run PyTorch Model resnet18_fx)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32235&view=logs&jobId=e3333e26-4334-5d5e-a7bd-ee7471860e42) | success | | ✅ | | [Model-Test (Run TensorFlow Model resnet50v1.5)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32235&view=logs&jobId=d570adab-99d2-59fe-b05d-1b608d431203) | success | | ✅ | | [Model-Test (Run TensorFlow Model ssd_resnet50_v1)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32235&view=logs&jobId=aff40fc6-1d0e-52aa-b4b9-bb1d69413565) | success | | ✅ | These checks are required after the changes to `neural_compressor/adaptor/pytorch.py`, `neural_compressor/adaptor/pytorch_cpu.yaml`, `neural_compressor/adaptor/torch_utils/bf16_convert.py`.
🟢 Unit Tests basic workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [UT-Basic](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234) | success | | ✅ | | [UT-Basic (Coverage Compare CollectDatafiles)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=71384379-497b-5787-1f51-cc2e0f831d78) | success | | ✅ | | [UT-Basic (Unit Test FWKs adaptor Test FWKs adaptor)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=e5896b99-a49d-517b-218b-3b918f0c116d) | success | | ✅ | | [UT-Basic (Unit Test FWKs adaptor baseline Test FWKs adaptor baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=f4046902-2b73-5016-25ba-63e28d81f503) | success | | ✅ | | [UT-Basic (Unit Test ITEX Test ITEX)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=7238057d-7331-5929-31ca-3e26743a41d0) | success | | ✅ | | [UT-Basic (Unit Test ITEX baseline Test ITEX baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=baf42d1a-8efd-5d70-9d09-b6bec5e8b55b) | success | | ✅ | | [UT-Basic (Unit Test Pruning Test PyTorch Pruning)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=99af828c-75c2-5dc7-0037-29447d6396f5) | success | | ✅ | | [UT-Basic (Unit Test Pruning Test TensorFlow Pruning)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=0a0ef4d6-07d3-59ba-0c8d-8c4daa447e58) | success | | ✅ | | [UT-Basic (Unit Test Pruning baseline Test PyTorch Pruning baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=b7474179-6386-5b4d-8836-092e0670eff4) | success | | ✅ | | [UT-Basic (Unit Test Pruning baseline Test TensorFlow Pruning baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=461a3442-9d84-5494-e465-40a939a41758) | success | | ✅ | | [UT-Basic (Unit Test TF newAPI Test TF newAPI)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=24d8ed3a-4c60-5ddd-e9bd-ea3ff52a4e98) | success | | ✅ | | [UT-Basic (Unit Test TF newAPI baseline Test TF newAPI baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=20c67c77-4f92-5283-357b-06b3e2359537) | success | | ✅ | | [UT-Basic (Unit Test User facing API Test User facing API)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=837bf053-a137-5cf0-a3a8-e85b8f58d512) | success | | ✅ | | [UT-Basic (Unit Test User facing API baseline Test User facing API baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=856f095c-c483-5936-5dd8-0c2f82743762) | success | | ✅ | | [UT-Basic (Unit Test other basic case Test other basic case)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=c6aa4c58-99e4-54e9-e3eb-cd322b75c938) | success | | ✅ | | [UT-Basic (Unit Test other cases baseline Test other cases baseline)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32234&view=logs&jobId=b62922ce-9ef8-5e77-08b8-9cffa1d355b8) | success | | ✅ |
UT-Basic coverage report
Base coverage PR coverage Diff
Lines 76.095% 76.101% 0.006%
Branches 65.171% 65.194% 0.023%
These checks are required after the changes to `neural_compressor/adaptor/pytorch.py`, `neural_compressor/adaptor/pytorch_cpu.yaml`, `neural_compressor/adaptor/torch_utils/bf16_convert.py`, `test/adaptor/pytorch_adaptor/test_adaptor_pytorch_2x.py`.
🟢 Unit Tests basic no coverage workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [UT-Basic-No-Coverage](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32236) | success | | ✅ | | [UT-Basic-No-Coverage (Unit Test FWKs adaptor Test FWKs adaptor)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32236&view=logs&jobId=e5896b99-a49d-517b-218b-3b918f0c116d) | success | | ✅ | | [UT-Basic-No-Coverage (Unit Test Pruning Test PyTorch Pruning)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32236&view=logs&jobId=99af828c-75c2-5dc7-0037-29447d6396f5) | success | | ✅ | | [UT-Basic-No-Coverage (Unit Test Pruning Test TensorFlow Pruning)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32236&view=logs&jobId=0a0ef4d6-07d3-59ba-0c8d-8c4daa447e58) | success | | ✅ | | [UT-Basic-No-Coverage (Unit Test User facing API Test User facing API)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32236&view=logs&jobId=837bf053-a137-5cf0-a3a8-e85b8f58d512) | success | | ✅ | | [UT-Basic-No-Coverage (Unit Test other basic case Test other basic case)](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32236&view=logs&jobId=c6aa4c58-99e4-54e9-e3eb-cd322b75c938) | success | | ✅ | These checks are required after the changes to `neural_compressor/adaptor/pytorch.py`, `neural_compressor/adaptor/pytorch_cpu.yaml`, `neural_compressor/adaptor/torch_utils/bf16_convert.py`, `test/adaptor/pytorch_adaptor/test_adaptor_pytorch_2x.py`.
🟢 Unit Tests ITREX workflow | Check ID | Status | Error details | | | -------- | ------ | ---- | --- | | [UT-ITREX](https://dev.azure.com/lpot-inc/b7121868-d73a-4794-90c1-23135f974d09/_build/results?buildId=32237) | success | | ✅ | These checks are required after the changes to `neural_compressor/adaptor/pytorch.py`, `neural_compressor/adaptor/pytorch_cpu.yaml`, `neural_compressor/adaptor/torch_utils/bf16_convert.py`.

Thank you for your contribution! 💜

Note This comment is automatically generated and will be updates every 180 seconds within the next 6 hours. If you have any other questions, contact chensuyue or XuehaoSun for help.