rtdetrv2导出模型失败

carter275 commented 1 week ago

问题确认 Search before asking

[X] 我已经查询历史issue，没有发现相似的bug。I have searched the issues and found no similar bug report.

Bug组件 Bug Component

Export

Bug描述 Describe the Bug

!python tools/export_model.py -c configs/rtdetrv2/rtdetrv2_r50vd_6x_coco.yml \ -o weights=https://bj.bcebos.com/v1/paddledet/models/rtdetrv2_r50vd_6x_coco.pdparams \ --output_dir=output_inference

[11/11 23:40:07] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/aistudio/.cache/paddle/weights/rtdetrv2_r50vd_6x_coco.pdparams [11/11 23:40:08] ppdet.data.source.category WARNING: anno_file 'dataset/coco/annotations/instances_val2017.json' is None or not set or not exist, please recheck TrainDataset/EvalDataset/TestDataset.anno_path, otherwise the default categories will be used by metric_type. [11/11 23:40:08] ppdet.data.source.category WARNING: metric_type: COCO, load default categories of COCO. [11/11 23:40:08] ppdet.engine INFO: Export inference config file to output_inference/rtdetrv2_r50vd_6x_coco/infer_cfg.yml Traceback (most recent call last): File "/home/aistudio/PaddleDetection-release-2.8/tools/export_model.py", line 118, in main() File "/home/aistudio/PaddleDetection-release-2.8/tools/export_model.py", line 114, in main run(FLAGS, cfg) File "/home/aistudio/PaddleDetection-release-2.8/tools/export_model.py", line 80, in run trainer.export(FLAGS.output_dir, for_fd=FLAGS.for_fd) File "/home/aistudio/PaddleDetection-release-2.8/ppdet/engine/trainer.py", line 1282, in export static_model, pruned_input_spec = self._get_infer_cfg_and_input_spec( File "/home/aistudio/PaddleDetection-release-2.8/ppdet/engine/trainer.py", line 1233, in _get_infer_cfg_and_input_spec input_spec, static_model.forward.main_program, File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1098, in main_program concrete_program = self.concrete_program File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 982, in concrete_program return self.concrete_program_specify_input_spec(input_spec=None) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1026, in concrete_program_specify_input_spec concreteprogram, = self.get_concrete_program( File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 914, in get_concrete_program concrete_program, partial_program_layer = self._program_cache[ File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1665, in getitem self._caches[item_id] = self._build_once(item) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1603, in _build_once concrete_program = ConcreteProgram.pir_from_func_spec( File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), kw) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/base/wrapped_decorator.py", line 40, in impl return wrapped_func(*args, *kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/base/dygraph/base.py", line 101, in impl return func(args, kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1276, in pir_from_func_spec error_data.raise_new_exception() File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/error.py", line 454, in raise_new_exception raise new_exception from None AssertionError: In transformed code:

File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/meta_arch.py", line 59, in forward
if self.training:
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/meta_arch.py", line 69, in forward
for inp in inputs_list:
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/meta_arch.py", line 76, in forward
outs.append(self.get_pred())
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/detr.py", line 118, in get_pred
return self._forward()
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/detr.py", line 89, in _forward
out_transformer = self.transformer(body_feats, pad_mask, self.inputs)
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformerv2.py", line 480, in forward
out_bboxes, out_logits = self.decoder(
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformer.py", line 235, in forward
output = layer(output, ref_points_input, memory,
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformerv2.py", line 259, in forward
tgt2 = self.cross_attn(
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformerv2.py", line 173, in forward
            format(reference_points.shape[-1]))

    output = self.ms_deformable_attn_core(value, value_spatial_shapes,
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
                                          sampling_locations, attention_weights)
    output = self.output_proj(output)

File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/utils.py", line 162, in deformable_attention_core_func_v2
value_list = [
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/utils.py", line 163, in <listcomp>
value.reshape([batch_num * num_heads, head_dim, h, w])
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/tensor/manipulation.py", line 4991, in reshape
new_shape = get_attr_shape(shape)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/tensor/manipulation.py", line 4937, in get_attr_shape
assert dim_size > 0, (

AssertionError: Each dimension value of 'shape' in reshape must not be negative except one unknown dimension. But received shape[0] = -8.

复现环境 Environment

paddle 3.0.0beta paddledetection 2.8

Bug描述确认 Bug description confirmation

[X] 我确认已经提供了Bug复现步骤、代码改动说明、以及环境信息，确认问题是可以复现的。I confirm that the bug replication steps, code change instructions, and environment information have been provided, and the problem can be reproduced.

是否愿意提交PR？ Are you willing to submit a PR?

[X] 我愿意提交PR！I'd like to help by submitting a PR!

kongxm889 commented 1 week ago

我也有一模一样的问题，感觉百度的东西太垃圾了，自己训练生成的模型自己都导不出来，简直不可理解。自己都不内测吗

Bobholamovic commented 1 week ago

看起来可能是模型不支持太动态的shape，请尝试在导出时添加-o TestReader.inputs_def.image_shape=[1,3,640,640]选项。

Bobholamovic commented 1 week ago

我也有一模一样的问题，感觉百度的东西太垃圾了，自己训练生成的模型自己都导不出来，简直不可理解。自己都不内测吗

很抱歉出现这样的问题，感谢你的反馈，我们会努力完善质量监测机制，避免之后再出现这样的问题。也希望来自开源社区的开发者如果感兴趣的话可以一同参与到项目的开发和维护中来，我们一起把这个仓库做好～

carter275 commented 1 week ago

看起来可能是模型不支持太动态的shape，请尝试在导出时添加-o TestReader.inputs_def.image_shape=[1,3,640,640]选项。

我直接把出错位置改为[1,3,640,640]`可以导出，但是没有pdmodel后缀的模型文件呢，而且其他模型导出也是只有这三个文件

Bobholamovic commented 1 week ago

从paddle 3.0-beta2版本开始，默认启用PIR，使用json格式代替pdmodel格式存储模型。所以这里的model.json其实就相当于以前的model.pdmodel～

carter275 commented 1 week ago

从paddle 3.0-beta2版本开始，默认启用PIR，使用json格式代替pdmodel格式存储模型。所以这里的model.json其实就相当于以前的model.pdmodel～

感谢您这么晚回复因为我是使用C++调用paddleinference进行推理部署的，我看C++的代码例子还是pdmodel格式呢，那么我该如何使用paddleinference调用现在的模型

Bobholamovic commented 1 week ago

我想应该：

使用最新的Paddle Inference C++ SDK来编译；
在需要传入pdmodel格式文件的地方，传入json格式文件。

对于第1点，可以在paddle官网下载；对于第2点，如果你使用的是PaddleDetection官方的C++推理代码的话，看起来目前这份代码还没有适配json格式模型的情况，可能需要修改这里。

Bobholamovic commented 1 week ago

另外一种选择是在导出模型时，使用旧版本的paddlepaddle，或者设置环境变量export FLAGS_enable_pir_api=0，以强制使用旧的pdmodel格式。

carter275 commented 1 week ago

旧版本的paddle是否影响redetrv2的使用呢

在 2024-11-13 00:05:12，"Lin Manhui" @.***> 写道：

另外一种选择是在导出模型时，使用旧版本的paddlepaddle，或者设置环境变量export FLAGS_enable_pir_api=0，以强制使用旧的pdmodel格式。

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

Bobholamovic commented 1 week ago

建议尝试使用paddle 2.6（最接近的版本）。

carter275 commented 1 week ago

好的，我试一下

在 2024-11-13 00:15:30，"Lin Manhui" @.***> 写道：

建议尝试使用paddle 2.6（最接近的版本）。

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

carter275 commented 1 week ago

今天试了，添加了也不能导出模型，麻烦解决一下

在 2024-11-12 17:15:47，"Lin Manhui" @.***> 写道：

看起来可能是模型不支持太动态的shape，请尝试在导出时添加-o TestReader.inputs_def.image_shape=[1,3,640,640]选项。

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

Bobholamovic commented 1 week ago

请问尝试的是哪一种方案呢

carter275 commented 1 week ago

TestReader.inputs_def.image_shape=[1,3,640,640] 这个方案导出，提示同样错误

在 2024-11-14 16:00:56，"Lin Manhui" @.***> 写道：

请问尝试的是哪一种方案呢

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

carter275 commented 1 week ago

请问尝试的是哪一种方案呢

!python tools/export_model.py -c configs/rtdetrv2/rtdetrv2_r50vd_6x_coco.yml \ -o weights=https://bj.bcebos.com/v1/paddledet/models/rtdetrv2_r50vd_6x_coco.pdparams trt=True TestReader.inputs_def.image_shape=[1,3,640,640] \ --output_dir=output_inference

结果如下 Warning: Unable to use numba in PP-Tracking, please install numba, for example(python3.7): pip install numba==0.56.4 Warning: Unable to use numba in PP-Tracking, please install numba, for example(python3.7): pip install numba==0.56.4 [11/15 00:39:52] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/aistudio/.cache/paddle/weights/rtdetrv2_r50vd_6x_coco.pdparams loading annotations into memory... Done (t=0.00s) creating index... index created! [11/15 00:39:52] ppdet.engine INFO: Export inference config file to output_inference/rtdetrv2_r50vd_6x_coco/infer_cfg.yml Traceback (most recent call last): File "/home/aistudio/PaddleDetection-release-2.8/tools/export_model.py", line 118, in main() File "/home/aistudio/PaddleDetection-release-2.8/tools/export_model.py", line 114, in main run(FLAGS, cfg) File "/home/aistudio/PaddleDetection-release-2.8/tools/export_model.py", line 80, in run trainer.export(FLAGS.output_dir, for_fd=FLAGS.for_fd) File "/home/aistudio/PaddleDetection-release-2.8/ppdet/engine/trainer.py", line 1282, in export static_model, pruned_input_spec = self._get_infer_cfg_and_input_spec( File "/home/aistudio/PaddleDetection-release-2.8/ppdet/engine/trainer.py", line 1233, in _get_infer_cfg_and_input_spec input_spec, static_model.forward.main_program, File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1062, in main_program concrete_program = self.concrete_program File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 941, in concrete_program return self.concrete_program_specify_input_spec(input_spec=None) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 986, in concrete_program_specify_input_spec concreteprogram, = self.get_concrete_program( File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 875, in get_concrete_program concrete_program, partial_program_layer = self._program_cache[ File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1648, in getitem self._caches[item_id] = self._build_once(item) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1575, in _build_once concrete_program = ConcreteProgram.from_func_spec( File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), kw) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/base/wrapped_decorator.py", line 26, in impl return wrapped_func(*args, *kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/base/dygraph/base.py", line 68, in impl return func(args, kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 1346, in from_func_spec error_data.raise_new_exception() File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/jit/dy2static/error.py", line 452, in raise_new_exception raise new_exception from None AssertionError: In transformed code:

File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/meta_arch.py", line 59, in forward
if self.training:
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/meta_arch.py", line 69, in forward
for inp in inputs_list:
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/meta_arch.py", line 76, in forward
outs.append(self.get_pred())
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/detr.py", line 118, in get_pred
return self._forward()
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/architectures/detr.py", line 89, in _forward
out_transformer = self.transformer(body_feats, pad_mask, self.inputs)
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformerv2.py", line 480, in forward
out_bboxes, out_logits = self.decoder(
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformer.py", line 235, in forward
output = layer(output, ref_points_input, memory,
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformerv2.py", line 259, in forward
tgt2 = self.cross_attn(
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/rtdetr_transformerv2.py", line 173, in forward
            format(reference_points.shape[-1]))

    output = self.ms_deformable_attn_core(value, value_spatial_shapes,
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
                                          sampling_locations, attention_weights)
    output = self.output_proj(output)

File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/utils.py", line 162, in deformable_attention_core_func_v2
value_list = [
File "/home/aistudio/PaddleDetection-release-2.8/ppdet/modeling/transformers/utils.py", line 163, in <listcomp>
value.reshape([batch_num * num_heads, head_dim, h, w])
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/tensor/manipulation.py", line 4017, in reshape
attrs["shape"] = get_attr_shape(shape)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/tensor/manipulation.py", line 3921, in get_attr_shape
assert dim_size > 0, (

AssertionError: Each dimension value of 'shape' in reshape must not be negative except one unknown dimension. But received shape[0] = -8.

Bobholamovic commented 1 week ago

看起来可能是模型不支持太动态的shape，请尝试在导出时添加-o TestReader.inputs_def.image_shape=[1,3,640,640]选项。

我直接把出错位置改为[1,3,640,640]`可以导出，但是没有pdmodel后缀的模型文件呢，而且其他模型导出也是只有这三个文件

想问下用这个方案可以导出不？

carter275 commented 6 days ago

用这个方案可以导出，但是使用paddleinference推理时失败

在 2024-11-15 01:48:56，"Lin Manhui" @.***> 写道：

看起来可能是模型不支持太动态的shape，请尝试在导出时添加-o TestReader.inputs_def.image_shape=[1,3,640,640]选项。

我直接把出错位置改为[1,3,640,640]`可以导出，但是没有pdmodel后缀的模型文件呢，而且其他模型导出也是只有这三个文件

想问下用这个方案可以导出不？

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

Bobholamovic commented 6 days ago

推理时报什么错误呢？

PaddlePaddle / PaddleDetection