[Question]: IndexError: (OutOfRange) The starting index 1 of slice is out of bounds in tensor 0-th axis, it shound be in the range of [-1, 1).

noobexplore commented 1 year ago

请提出你的问题

UIE在运行finetune完成之后，测试模型压缩时候出现以下报错：

[2022-12-05 16:01:12,170] [    INFO] - Inference model exported.
Traceback (most recent call last):
  File "/home/xtl/github_download/PaddleNLP/model_zoo/uie/finetune.py", line 288, in <module>
    main()
  File "/home/xtl/github_download/PaddleNLP/model_zoo/uie/finetune.py", line 284, in main
    trainer.compress(custom_evaluate=custom_evaluate)
  File "/home/xtl/github_download/PaddleNLP/paddlenlp/trainer/trainer_compress.py", line 65, in compress
    _dynabert(self, self.model, args.output_dir)
  File "/home/xtl/github_download/PaddleNLP/paddlenlp/trainer/trainer_compress.py", line 132, in _dynabert
    ofa_model, teacher_model = _dynabert_init(self, model, eval_dataloader)
  File "/home/xtl/github_download/PaddleNLP/paddlenlp/trainer/trainer_compress.py", line 249, in _dynabert_init
    teacher_model = copy.deepcopy(model)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 296, in _reconstruct
    value = deepcopy(value, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/xtl/anaconda3/envs/docker/lib/python3.8/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle 'paddle.fluid.core_avx.BlockDesc' object

大概看了一下错误信息发现错误定位到在初始化dynabert中deepcopy教师模型的时候，不知道为啥会出现这样的错误，相关paddle版本如下：

paddle-bfloat        0.1.7
paddle2onnx          1.0.1
paddlefsl            1.1.0
paddlenlp            2.4.4
paddlepaddle-gpu     2.3.2.post112
paddleslim           2.3.4
paddleslim-opt-tools 2.11.2

LiuChiachi commented 1 year ago

您好，您的model是如何定义的？可以在deepcopy之前print(model)看一下

noobexplore commented 1 year ago

@LiuChiachi 你好，这里的model定义就是paddlenlp中modelzoo里面提供的UIE模型，就是跑示例的代码最后进行模型压缩的时候出现以上的问题

LiuChiachi commented 1 year ago

之前有进行量化训练QAT吗

noobexplore commented 1 year ago

之前有进行量化训练QAT吗

没有，是需要先进行量化训练么，我就是在测试给出的示例代码中有在最后训练完成之后进行模型压缩。这块有相关参考示例么？

noobexplore commented 1 year ago

之前有进行量化训练QAT吗

因为我看示例代码中就是在训练完成之后，将模型传入到下面的方法进行初始化了：

def _dynabert_init(self, model, eval_dataloader):
    from paddleslim.nas.ofa.convert_super import Convert, supernet
    from paddleslim.nas.ofa import OFA, DistillConfig, utils

    # Step1: Initialize a dictionary to save the weights from the origin model.
    origin_weights = model.state_dict()

    # Step2: Define teacher model.
    teacher_model = copy.deepcopy(model)

    # Step3: Convert origin model to supernet.
    sp_config = supernet(expand_ratio=[1.0])
    model = Convert(sp_config).convert(model)
    ......

就是在step2的时候发现了错误，意思是这里传入的模型是需要先量化训练过的模型么？

LiuChiachi commented 1 year ago

这里的model需要是原始的FP32 模型，不能是量化后的

noobexplore commented 1 year ago

@LiuChiachi 你好，我更新到最新版本之后已经没有上述的问题，目前的版本信息如下：

paddlenlp 2.4.8
paddlepaddle-gpu 2.4.1.post112
paddleslim 2.4.1

现在进行UIE压缩过程中，在压缩策略"dynabert+ptq"下老是会报以下错误：

[2022-12-30 13:37:13,517] [    INFO] - Teacher's evaluation starts.
Traceback (most recent call last):
  File "finetune.py", line 245, in <module>
    main()
  File "finetune.py", line 241, in main
    trainer.compress(custom_evaluate=custom_evaluate)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddlenlp/trainer/trainer_compress.py", line 70, in compress
    _dynabert(self, self.model, args.output_dir)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddlenlp/trainer/trainer_compress.py", line 138, in _dynabert
    ofa_model = _dynabert_training(
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddlenlp/trainer/trainer_compress.py", line 407, in _dynabert_training
    evaluate(self, teacher_model, eval_dataloader)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddlenlp/trainer/trainer_compress.py", line 304, in evaluate
    return self.custom_evaluate(self, model, data_loader)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 375, in _decorate_function
    return func(*args, **kwargs)
  File "finetune.py", line 226, in custom_evaluate
    logits = model(
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 948, in __call__
    return self.forward(*inputs, **kwargs)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddlenlp/transformers/ernie/modeling.py", line 1257, in forward
    sequence_output, _ = self.ernie(
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 948, in __call__
    return self.forward(*inputs, **kwargs)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddlenlp/trainer/trainer_compress.py", line 800, in auto_model_dynabert_forward
    encoder_outputs = self.encoder(embedding_output, **encoder_kwargs)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 948, in __call__
    return self.forward(*inputs, **kwargs)
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddlenlp/transformers/ofa_utils.py", line 123, in encoder_ofa_forward
    output = mod(output, src_mask=[src_mask[0], head_mask[i]])
  File "/home/xtl/anaconda3/envs/pp_com/lib/python3.8/site-packages/paddle/fluid/dygraph/varbase_patch_methods.py", line 753, in __getitem__
    return self._getitem_index_not_tensor(item)
IndexError: (OutOfRange) The starting index 1 of slice is out of bounds in tensor 0-th axis, it shound be in the range of [-1, 1). (at /paddle/paddle/fluid/pybind/slice_utils.h:209)

不知道到是什么问题，然后我参考其他issue中问题将策略改为量化"qat"就可以正常的压缩到对应的文件夹下。

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动，被标记为stale。

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天，即将关闭。

PaddlePaddle / PaddleNLP

[Question]: IndexError: (OutOfRange) The starting index 1 of slice is out of bounds in tensor 0-th axis, it shound be in the range of [-1, 1). #4016

请提出你的问题