[Bug] Cannot export ESPNetV2 models

ukoehler commented 1 year ago

System: Windows 10 Python 3.9.13 paddle-bfloat 0.1.7 paddle2onnx 1.0.1 paddlefsl 1.1.0 paddlehub 2.3.0 paddlenlp 2.4.1 paddlepaddle 2.3.2 paddleseg 2.6.0

Executing the following command to export ESPNetV2:

(.venv) D:\Local\devel\Python\PaddleSeg\PaddleSeg>python export.py --config conf
igs/espnet/espnet_cityscapes_1024x512_120k.yml --model_path ESPNetV2/model.pdpar
ams --save_dir ESPNetV2 --without_argmax --with_softmax --input_shape 1 3 1024 5
12

leads to the following error message:

2022-11-17 14:38:52 [INFO]      Loaded trained params of model successfully.
Traceback (most recent call last):
  File "D:\Local\devel\Python\PaddleSeg\PaddleSeg\export.py", line 144, in <modu
le>
    main(args)
  File "D:\Local\devel\Python\PaddleSeg\PaddleSeg\export.py", line 123, in main
    paddle.jit.save(new_net, save_path)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\jit.py", line 631, in wrapper
    func(layer, path, input_spec, **configs)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\decorator.py", l
ine 232, in fun
    return caller(func, *(extras + args), **kw)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\wra
pped_decorator.py", line 25, in __impl__
    return wrapped_func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\base.py", line 51, in __impl__
    return func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\jit.py", line 860, in save
    concrete_program = static_func.concrete_program_specify_input_spec(
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\dygraph_to_static\program_translator.py", line 527, in concrete_program_spe
cify_input_spec
    concrete_program, _ = self.get_concrete_program(
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\dygraph_to_static\program_translator.py", line 436, in get_concrete_program

    concrete_program, partial_program_layer = self._program_cache[cache_key]
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\dygraph_to_static\program_translator.py", line 801, in __getitem__
    self._caches[item_id] = self._build_once(item)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\dygraph_to_static\program_translator.py", line 785, in _build_once
    concrete_program = ConcreteProgram.from_func_spec(
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\decorator.py", l
ine 232, in fun
    return caller(func, *(extras + args), **kw)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\wra
pped_decorator.py", line 25, in __impl__
    return wrapped_func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\base.py", line 51, in __impl__
    return func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\dygraph_to_static\program_translator.py", line 740, in from_func_spec
    error_data.raise_new_exception()
  File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\dyg
raph\dygraph_to_static\error.py", line 336, in raise_new_exception
    six.exec_("raise new_exception from None")
  File "<string>", line 1, in <module>
ValueError: In transformed code:

    File "D:\Local\devel\Python\PaddleSeg\PaddleSeg\export.py", line 74, in forw
ard
        outs = self.net(x)
    File "D:\Local\devel\Python\PaddleSeg\PaddleSeg\paddleseg\models\espnet.py",
 line 103, in forward
        out_l1, out_l2, out_l3, out_l4 = self.backbone(x)
    File "D:\Local\devel\Python\PaddleSeg\PaddleSeg\paddleseg\models\espnet.py",
 line 451, in forward
        out_l2 = self.level2(out_l1, x)
    File "D:\Local\devel\Python\PaddleSeg\PaddleSeg\paddleseg\models\espnet.py",
 line 350, in forward
            # import pdb
            # pdb.set_trace()
            output = output + self.shortcut_layer(inputs)
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        return self._act(output)

    File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\l
ayers\math_op_patch.py", line 342, in __impl__
        current_block(self).append_op(
    File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\f
ramework.py", line 3615, in append_op
        op = Operator(
    File "D:\Local\devel\Python\PaddleSeg\.venv\lib\site-packages\paddle\fluid\f
ramework.py", line 2764, in __init__
        self.desc.infer_shape(self.block.desc)

    ValueError: (InvalidArgument) Broadcast dimension mismatch. Operands could n
ot be broadcast together with the shape of X = [1, 128, 256, 128] and the shape
of Y = [1, 128, 512, 256]. Received [256] in X is not equal to [512] in Y at i:2
.
  [Hint: Expected x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 ||
y_dims_array[i] <= 1 == true, but received x_dims_array[i] == y_dims_array[i] ||
 x_dims_array[i] <= 1 || y_dims_array[i] <= 1:0 != true:1.] (at C:\home\workspac
e\Paddle_release\paddle/phi/kernels/funcs/common_shape.h:84)
  [operator < elementwise_add > error]

shiyutang commented 1 year ago

Please upgrade the paddle to develop and try again. install guide: https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html

ukoehler commented 1 year ago

Hmmm, disc is almost full and I do not speak Chinese. Is there an English version of the web page?

shiyutang commented 1 year ago

yes，https://www.paddlepaddle.org.cn/en

ukoehler commented 1 year ago

No changes for version 2.7.0

shiyutang commented 1 year ago

It works if you do not set the input shape.

ukoehler commented 1 year ago

I first tried to infer images with version 2.7.0: Fails

(.venv) D:\Local\devel\Python\PaddleSeg_2.7.0>python predict.py --config configs
/espnet/espnet_cityscapes_1024x512_120k.yml --model_path ESPNetV2/model.pdparams
 --image_path "D:\Local\devel\Python\TestImages\Input" --save_dir ESPNetV2
2023-01-13 13:28:23 [INFO]
---------------Config Information---------------
batch_size: 8
iters: 120000
loss:
  coef:
  - 1
  - 1
  types:
  - type: CrossEntropyLoss
    weight:
    - 2.79834108
    - 6.92945723
    - 3.84068512
    - 9.94349362
    - 9.77098823
    - 9.51484
    - 10.30981624
    - 9.94307377
    - 4.64933892
    - 9.55759938
    - 7.86692178
    - 9.53126629
    - 10.3496365
    - 6.67234062
    - 10.26054204
    - 10.28785275
    - 10.28988296
    - 10.40546021
    - 10.13848367
lr_scheduler:
  end_lr: 0.0
  learning_rate: 0.001
  power: 0.9
  type: PolynomialDecay
model:
  drop_prob: 0.0
  in_channels: 3
  num_classes: 19
  scale: 2.0
  type: ESPNetV2
optimizer:
  type: adam
  weight_decay: 0.0002
train_dataset:
  dataset_root: data/cityscapes
  mode: train
  transforms:
  - max_scale_factor: 2.0
    min_scale_factor: 0.5
    scale_step_size: 0.25
    type: ResizeStepScaling
  - crop_size:
    - 1024
    - 512
    type: RandomPaddingCrop
  - type: RandomHorizontalFlip
  - brightness_range: 0.4
    contrast_range: 0.4
    saturation_range: 0.4
    type: RandomDistort
  - type: Normalize
  type: Cityscapes
val_dataset:
  dataset_root: data/cityscapes
  mode: val
  transforms:
  - type: Normalize
  type: Cityscapes
------------------------------------------------
2023-01-13 13:28:23 [INFO]      Number of predict images = 120
2023-01-13 13:28:23 [INFO]      Loading pretrained model from ESPNetV2/model.pdp
arams
2023-01-13 13:28:23 [INFO]      There are 365/365 variables loaded into ESPNetV2
.
2023-01-13 13:28:23 [INFO]      Start to predict...
Traceback (most recent call last):
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\predict.py", line 191, in <module>
    main(args)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\predict.py", line 176, in main
    predict(
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\core\predict.py", line 1
15, in predict
    pred, _ = infer.inference(
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\core\infer.py", line 158
, in inference
    logits = model(im)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\layers.py", line 948, in __call__
    return self.forward(*inputs, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\models\espnet.py", line
114, in forward
    merged_l2 = self.project_l2(paddle.concat([out_l2, l3_to_l2], axis=1))
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\ten
sor\manipulation.py", line 1140, in concat
    return _C_ops.concat(input, axis)
ValueError: (InvalidArgument) The 3-th dimension of input[0] and input[1] is exp
ected to be equal.But received input[0]'s shape = [1, 128, 160, 147], input[1]'s
 shape = [1, 19, 160, 148].
  [Hint: Expected inputs_dims[0][j] == inputs_dims[i][j], but received inputs_di
ms[0][j]:147 != inputs_dims[i][j]:148.] (at C:\home\workspace\Paddle\paddle/phi/
kernels/funcs/concat_funcs.h:83)

Tried to export with input size: Fails

(.venv) D:\Local\devel\Python\PaddleSeg_2.7.0>python export.py --config configs/
espnet/espnet_cityscapes_1024x512_120k.yml --model_path ESPNetV2/model.pdparams
--save_dir ESPNetV2 --without_argmax --with_softmax --input_shape 1 3 1024 512
2023-01-13 13:30:46 [INFO]      Loaded trained params of model successfully.
2023-01-13 13:30:46 [WARNING]   --without_argmax will be deprecated, please use
--output_op
2023-01-13 13:30:46 [WARNING]   --with_softmax will be deprecated, please use --
output_op
<frozen importlib._bootstrap>:283: DeprecationWarning: the load_module() method
is deprecated and slated for removal in Python 3.12; use exec_module() instead
D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\fluid\layer
s\control_flow.py:1359: UserWarning: In dy2static mode, we attemp to assign a va
riable with shape (1, 3, 512, 256) into a variable with shape(1, 3, 1024, 512),
which is not always right.
  warnings.warn(
Traceback (most recent call last):
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\export.py", line 143, in <module>
    main(args)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\export.py", line 115, in main
    paddle.jit.save(new_net, save_path)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\decorator.
py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\wrapped_decorator.py", line 26, in __impl__
    return wrapped_func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\jit.py", line 649, in wrapper
    func(layer, path, input_spec, **configs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\decorator.
py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\wrapped_decorator.py", line 26, in __impl__
    return wrapped_func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\base.py", line 67, in __impl__
    return func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\jit.py", line 927, in save
    concrete_program = static_func.concrete_program_specify_input_spec(
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\dygraph_to_static\program_translator.py", line 577, in concrete_progr
am_specify_input_spec
    concrete_program, _ = self.get_concrete_program(
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\dygraph_to_static\program_translator.py", line 485, in get_concrete_p
rogram
    concrete_program, partial_program_layer = self._program_cache[cache_key]
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\dygraph_to_static\program_translator.py", line 955, in __getitem__
    self._caches[item_id] = self._build_once(item)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\dygraph_to_static\program_translator.py", line 939, in _build_once
    concrete_program = ConcreteProgram.from_func_spec(
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\decorator.
py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\wrapped_decorator.py", line 26, in __impl__
    return wrapped_func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\base.py", line 67, in __impl__
    return func(*args, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\dygraph_to_static\program_translator.py", line 895, in from_func_spec
    error_data.raise_new_exception()
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\dygraph_to_static\error.py", line 350, in raise_new_exception
    six.exec_("raise new_exception from None")
  File "<string>", line 1, in <module>
ValueError: In transformed code:

    File "D:\Local\devel\Python\PaddleSeg_2.7.0\export.py", line 68, in forward
        outs = self.net(x)
    File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\models\espnet.py", lin
e 103, in forward
        out_l1, out_l2, out_l3, out_l4 = self.backbone(x)
    File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\models\espnet.py", lin
e 451, in forward
        out_l2 = self.level2(out_l1, x)
    File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\models\espnet.py", lin
e 340, in forward
        if inputs is not None:
    File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\models\espnet.py", lin
e 350, in forward
            # import pdb
            # pdb.set_trace()
            output = output + self.shortcut_layer(inputs)
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        return self._act(output)

    File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\f
luid\layers\math_op_patch.py", line 417, in __impl__
        current_block(self).append_op(type=op_type,
    File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\f
luid\framework.py", line 4017, in append_op
        op = Operator(
    File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\f
luid\framework.py", line 3016, in __init__
        self.desc.infer_shape(self.block.desc)

    ValueError: (InvalidArgument) Broadcast dimension mismatch. Operands could n
ot be broadcast together with the shape of X = [1, 128, 256, 128] and the shape
of Y = [1, 128, 512, 256]. Received [256] in X is not equal to [512] in Y at i:2
.
  [Hint: Expected x_dims_array[i] == y_dims_array[i] || x_dims_array[i] <= 1 ||
y_dims_array[i] <= 1 == true, but received x_dims_array[i] == y_dims_array[i] ||
 x_dims_array[i] <= 1 || y_dims_array[i] <= 1:0 != true:1.] (at C:\home\workspac
e\Paddle\paddle/phi/kernels/funcs/common_shape.h:84)
  [operator < elementwise_add > error]

Export without input size and export to ONNX: crashes

(.venv) D:\Local\devel\Python\PaddleSeg_2.7.0>paddle2onnx --model_dir ESPNetV2 -
-model_filename model.pdmodel --params_filename model.pdiparams --opset_version
13 --save_file ESPNetV2/espnet_cityscapes_1024x512_120k.onnx
[Paddle2ONNX] Start to parse PaddlePaddle model...
[Paddle2ONNX] Model file path: ESPNetV2\model.pdmodel
[Paddle2ONNX] Paramters file path: ESPNetV2\model.pdiparams
[Paddle2ONNX] Start to parsing Paddle model...
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_1.tmp_0] While dilation
s != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_2.tmp_0] While dilation
s != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_3.tmp_0] While dilation
s != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[Paddle2ONNX] Detected there's control flow 'while' op in your model, this requi
res the minimal opset version of 13.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_5.tmp_0] While dilation
s != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_6.tmp_0] While dilation
s != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_7.tmp_0] While dilation
s != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[Paddle2ONNX] Detected there's control flow 'while' op in your model, this requi
res the minimal opset version of 13.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_9.tmp_0] While dilation
s != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_10.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_11.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_13.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_14.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_15.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_17.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_18.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_19.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_21.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_22.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_23.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[Paddle2ONNX] Detected there's control flow 'while' op in your model, this requi
res the minimal opset version of 13.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_26.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_27.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_30.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_31.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_34.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_35.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_38.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_39.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_42.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_43.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_46.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_47.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_50.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_51.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_54.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR][Paddle2ONNX] [depthwise_conv2d: depthwise_conv2d_55.tmp_0] While dilatio
ns != 1, cannot support padding = 'SAME'.
[Paddle2ONNX] Due to the operator: depthwise_conv2d, this model cannot be export
ed to ONNX.
[ERROR] Model exporting failed, you can report this problem to https://github.co
m/PaddlePaddle/Paddle2ONNX.git.

shiyutang commented 1 year ago

The paddle2onnx issue is because they did not support padding='same' in depthewise_conv2d, you can try put forward an issue in paddle2onnx: https://github.com/PaddlePaddle/Paddle2ONNX.
The export problem: it is a bug in dy2static, and we have reported it for fix.
The predict problem: After evaluation on develop branch, we can not reproduce the problem during infer on https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png. It is likely this is a problem caused by different image shapes, we recommend you provide a sample image for us to test

ukoehler commented 1 year ago

issue submitted
great, waiting for it
The image Matting/demo/human.jpg does predict. Slightly resized, however, it will not:

(.venv) D:\Local\devel\Python\PaddleSeg_2.7.0>python predict.py --config configs
/espnet/espnet_cityscapes_1024x512_120k.yml --model_path ESPNetV2/model.pdparams
 --image_path "D:\Local\devel\Python\TestImages\Input\human_resized.png" --save_
dir ESPNetV2
2023-02-21 10:06:21 [INFO]
---------------Config Information---------------
batch_size: 8
iters: 120000
loss:
  coef:
  - 1
  - 1
  types:
  - type: CrossEntropyLoss
    weight:
    - 2.79834108
    - 6.92945723
    - 3.84068512
    - 9.94349362
    - 9.77098823
    - 9.51484
    - 10.30981624
    - 9.94307377
    - 4.64933892
    - 9.55759938
    - 7.86692178
    - 9.53126629
    - 10.3496365
    - 6.67234062
    - 10.26054204
    - 10.28785275
    - 10.28988296
    - 10.40546021
    - 10.13848367
lr_scheduler:
  end_lr: 0.0
  learning_rate: 0.001
  power: 0.9
  type: PolynomialDecay
model:
  drop_prob: 0.0
  in_channels: 3
  num_classes: 19
  scale: 2.0
  type: ESPNetV2
optimizer:
  type: adam
  weight_decay: 0.0002
train_dataset:
  dataset_root: data/cityscapes
  mode: train
  transforms:
  - max_scale_factor: 2.0
    min_scale_factor: 0.5
    scale_step_size: 0.25
    type: ResizeStepScaling
  - crop_size:
    - 1024
    - 512
    type: RandomPaddingCrop
  - type: RandomHorizontalFlip
  - brightness_range: 0.4
    contrast_range: 0.4
    saturation_range: 0.4
    type: RandomDistort
  - type: Normalize
  type: Cityscapes
val_dataset:
  dataset_root: data/cityscapes
  mode: val
  transforms:
  - type: Normalize
  type: Cityscapes
------------------------------------------------
2023-02-21 10:06:21 [INFO]      Number of predict images = 1
2023-02-21 10:06:21 [INFO]      Loading pretrained model from ESPNetV2/model.pdp
arams
2023-02-21 10:06:21 [INFO]      There are 365/365 variables loaded into ESPNetV2
.
2023-02-21 10:06:21 [INFO]      Start to predict...
Traceback (most recent call last):
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\predict.py", line 191, in <module>
    main(args)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\predict.py", line 176, in main
    predict(
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\core\predict.py", line 1
15, in predict
    pred, _ = infer.inference(
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\core\infer.py", line 158
, in inference
    logits = model(im)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\flu
id\dygraph\layers.py", line 948, in __call__
    return self.forward(*inputs, **kwargs)
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\paddleseg\models\espnet.py", line
108, in forward
    merged_l3 = self.eesp_psp(paddle.concat([out_l3, l4_to_l3], axis=1))
  File "D:\Local\devel\Python\PaddleSeg_2.7.0\.venv\lib\site-packages\paddle\ten
sor\manipulation.py", line 1140, in concat
    return _C_ops.concat(input, axis)
ValueError: (InvalidArgument) The 2-th dimension of input[0] and input[1] is exp
ected to be equal.But received input[0]'s shape = [1, 256, 65, 125], input[1]'s
shape = [1, 256, 66, 126].
  [Hint: Expected inputs_dims[0][j] == inputs_dims[i][j], but received inputs_di
ms[0][j]:65 != inputs_dims[i][j]:66.] (at C:\home\workspace\Paddle\paddle/phi/ke
rnels/funcs/concat_funcs.h:83)

shiyutang commented 1 year ago

I am not sure what do you mean in your third question.

ukoehler commented 1 year ago

It's not a question. Rather help for you. You guessed it might be related to image size and I have now given you a picture that works and one that does not work when resized. Including the resized image (the original is in the repository) and the error message generated when processing the resized image.

shiyutang commented 1 year ago

Thank you. We will update you as soon as the bug is fixed.

shiyutang commented 1 year ago

Please see https://github.com/PaddlePaddle/PaddleSeg/pull/3003 and the bug is fixed. Btw, we assigned input shape in export process. Therefore it is necessary to keep the image shape the same as the export shape during prediction to get the correct result.

shiyutang commented 8 months ago

This bug has been fixed.

PaddlePaddle / PaddleSeg

[Bug] Cannot export ESPNetV2 models #2732