PaddlePaddle / Paddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (『飞桨』核心框架,深度学习&机器学习高性能单机、分布式训练和跨平台部署)
http://www.paddlepaddle.org/
Apache License 2.0
22.11k stars 5.55k forks source link

[bug] 多个预测模型,基于 PaddleSlim 使用多卡启动 自动压缩产出量化模型报错:IndexError: list index out of range #61626

Closed EmmonsCurse closed 7 months ago

EmmonsCurse commented 7 months ago

bug描述 Describe the Bug

问题描述

基于 develop 分支编包,EfficientNetB0、GhostNet_x1_0 与 MobileNetV1 等多个预测模型,多卡启动 自动压缩产出量化模型报错:IndexError: list index out of range

Traceback (most recent call last):
  File "run.py", line 212, in <module>
    main()
  File "run.py", line 205, in main
    ac.compress()
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddleslim-0.0.0.dev0-py3.8.egg/paddleslim/auto_compression/compressor.py", line 586, in compress
    self.single_strategy_compress(strategy, config, strategy_idx,
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddleslim-0.0.0.dev0-py3.8.egg/paddleslim/auto_compression/compressor.py", line 769, in single_strategy_compress
    train_program_info, test_program_info = self._prepare_program(
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddleslim-0.0.0.dev0-py3.8.egg/paddleslim/auto_compression/compressor.py", line 519, in _prepare_program
    train_program_info, test_program_info, self._quant_config = build_quant_program(
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddleslim-0.0.0.dev0-py3.8.egg/paddleslim/auto_compression/create_compressed_program.py", line 432, in build_quant_program
    train_program = quant_aware(
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddleslim-0.0.0.dev0-py3.8.egg/paddleslim/quant/quanter.py", line 456, in quant_aware
    transform_pass.apply(sub_graph)
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddle/static/quantization/quantization_pass.py", line 2787, in apply
    self._transform_forward(graph, op)
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddle/static/quantization/quantization_pass.py", line 2562, in _transform_forward
    var_node = self._insert_func(
  File "/opt/_internal/cpython-3.8.0/lib/python3.8/site-packages/paddle/static/quantization/quantization_pass.py", line 994, in _insert_func
    op_grad = op_out_grad.outputs[0]
IndexError: list index out of range

环境信息

复现方法

可参照 https://github.com/PaddlePaddle/PaddleSlim/tree/develop/example/auto_compression/image_classification

准备环境

# 安装 paddlepaddle:
wget https://paddle-qa.bj.bcebos.com/paddle-pipeline/Develop-GpuAll-LinuxCentos-Gcc82-Cuda112-Trtoff-Py38-Compile/latest/paddlepaddle_gpu-0.0.0-cp38-cp38-linux_x86_64.whl
python -m pip install paddlepaddle_gpu-0.0.0-cp38-cp38-linux_x86_64.whl

# 安装 develop 版本 paddleslim
git clone https://github.com/PaddlePaddle/PaddleSlim.git -b develop

cd PaddleSlim
python setup.py install
python -m pip install -r requirements.txt

# 数据集准备
cd ./example/auto_compression/image_classification

wget https://paddle-qa.bj.bcebos.com/PaddleClas/ILSVRC2012_data_demo.tar
tar xf ILSVRC2012_data_demo.tar
mv ILSVRC2012_data_demo ILSVRC2012

# 模型准备
## 多个下载
# root_url="https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference"
# 
# models="EfficientNetB0 GhostNet_x1_0 MobileNetV1 \ 
#         MobileNetV3_large_x1_0 PPHGNet_tiny PPLCNet_x1_0 PPLCNetV2_base"
# 
# for model in ${models}
# do
#     if [ ! -f ${model}_infer.tar ]; then
#         echo ---infer ${model} downloading-----
#         wget -q ${root_url}/${model}_infer.tar
#         tar xf ${model}_infer.tar
#     fi
# done

## 单个下载
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/EfficientNetB0_infer.tar
tar -xf EfficientNetB0_infer.tar

执行多卡启动 自动压缩并产出量化模型

export CUDA_VISIBLE_DEVICES=0,1
export PADDLE_LOG_LEVEL=debug

# 修改模型名 ${model}  执行
python -m paddle.distributed.launch run.py --save_dir=./save_quant/EfficientNetB0_act_qat/ --config_path=./configs/EfficientNetB0/qat_dis.yaml

其他补充信息 Additional Supplementary Information

@zzjjay @vivienfanghuagood 辛苦看一下,谢谢。

zzjjay commented 7 months ago

这个问题已提pr修复,https://github.com/PaddlePaddle/PaddleSlim/pull/1848