marshall-dteach / SwinUNet

基于SwinUNet论文复现
Apache License 2.0
16 stars 0 forks source link

SwinUnet可以使用[1,512,512]训练吗? #2

Open SurvivorNo1 opened 1 year ago

SurvivorNo1 commented 1 year ago

作者您好, 我正在Medicalseg上使用您基于paddlepaddle复现的模型SwinUnet, 但是我希望以更大的尺寸[1,512,512]进行训练,而不是以[1,224,224].我直接修改了配置文件中 transforms-Resize3D的size参数,但是不能正常工作。报错信息如下。所以我想寻求您的帮助,如果使用[1,512,512]进行训练,需要在您的代码里做怎样的参数修改呢?

AiStudio环境:

硬件信息
CPU:4核
AI加速卡:v100
总显存:32GB
总内存:32 GB
总硬盘:100 GB
环境配置
Python:3.7.4
框架信息
框架版本:PaddlePaddle 2.3.2

配置文件:

data_root: /tmp/data

train_dataset:
  type: Abdomen
  dataset_root: abdomen/abdomen_phase0
  result_dir: abdomen/abdomen_phase1
  transforms:
    - type: Resize3D
      size: [1, 512, 512]
  mode: train
  num_classes: 16

val_dataset:
  type: Abdomen
  dataset_root: abdomen/abdomen_phase0
  result_dir: abdomen/abdomen_phase1
  num_classes: 16
  transforms:
    - type: Resize3D
      size: [1, 512, 512]
  mode: val
  dataset_json_path: data/abdomen/abdomen_raw/dataset.json

test_dataset:
  type: Abdomen
  dataset_root:  abdomen/abdomen_phase0
  result_dir:  abdomen/abdomen_phase1
  num_classes: 16
  transforms:
    - type: Resize3D
      size: [1, 512, 512]
  mode: val
  dataset_json_path: data/abdomen/abdomen_raw/dataset.json

batch_size: 48
iters: 100000

model:
  type: SwinUNet
  backbone:
    type: SwinTransformer_tinier_patch4_window7_224
  num_classes: 16

optimizer:
  type: sgd
  momentum: 0.9
  weight_decay: 1.0e-4

lr_scheduler:
  learning_rate: 0.10
  type: PolynomialDecay
  end_lr: 0
  power: 0.9

loss:
  types:
    - type: MixedLoss
      losses:
        - type: CrossEntropyLoss
        - type: DiceLoss
      coef: [0.4, 0.6]
  coef: [1]

export:
  transforms:
    - type: Resize3D
      size: [1, 512, 512]
  inference_helper:
    type: InferenceHelper2D

报错信息:

Traceback (most recent call last):
  File "train.py", line 232, in <module>
    main(args)
  File "train.py", line 227, in main
    has_dataset_json=args.has_dataset_json)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/core/train.py", line 138, in train
    logits_list = model(images)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 200, in forward
    x, x_downsample = self.backbone(x)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 670, in forward
    out = self.forward_features(x)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 662, in forward_features
    x = layer(x)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 481, in forward
    x = blk(x)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 329, in forward
    assert L == H * W, "input feature has wrong size"
AssertionError: input feature has wrong size
marshall-dteach commented 1 year ago

应该是SwinUNet的配置中的输入img_size会固定成224,修改的话要对应修改(我记得应该是位置编码啥的有关系)

------------------ 原始邮件 ------------------ 发件人: "marshall-dteach/SwinUNet" @.>; 发送时间: 2023年4月15日(星期六) 晚上8:47 @.>; @.***>; 主题: [marshall-dteach/SwinUNet] SwinUnet可以使用[1,512,512]训练吗? (Issue #2)

作者您好, 我正在Medicalseg上使用您基于paddlepaddle复现的模型SwinUnet, 但是我希望以更大的尺寸[1,512,512]进行训练,而不是以[1,224,224].我直接修改了配置文件中 transforms-Resize3D的size参数,但是不能正常工作。报错信息如下。所以我想寻求您的帮助,如果使用[1,512,512]进行训练,需要在您的代码里做怎样的参数修改呢?

AiStudio环境: 硬件信息 CPU:4核 AI加速卡:v100 总显存:32GB 总内存:32 GB 总硬盘:100 GB 环境配置 Python:3.7.4 框架信息 框架版本:PaddlePaddle 2.3.2

配置文件: data_root: /tmp/data train_dataset: type: Abdomen dataset_root: abdomen/abdomen_phase0 result_dir: abdomen/abdomen_phase1 transforms: - type: Resize3D size: [1, 512, 512] mode: train num_classes: 16 val_dataset: type: Abdomen dataset_root: abdomen/abdomen_phase0 result_dir: abdomen/abdomen_phase1 num_classes: 16 transforms: - type: Resize3D size: [1, 512, 512] mode: val dataset_json_path: data/abdomen/abdomen_raw/dataset.json test_dataset: type: Abdomen dataset_root: abdomen/abdomen_phase0 result_dir: abdomen/abdomen_phase1 num_classes: 16 transforms: - type: Resize3D size: [1, 512, 512] mode: val dataset_json_path: data/abdomen/abdomen_raw/dataset.json batch_size: 48 iters: 100000 model: type: SwinUNet backbone: type: SwinTransformer_tinier_patch4_window7_224 num_classes: 16 optimizer: type: sgd momentum: 0.9 weight_decay: 1.0e-4 lr_scheduler: learning_rate: 0.10 type: PolynomialDecay end_lr: 0 power: 0.9 loss: types: - type: MixedLoss losses: - type: CrossEntropyLoss - type: DiceLoss coef: [0.4, 0.6] coef: [1] export: transforms: - type: Resize3D size: [1, 512, 512] inference_helper: type: InferenceHelper2D
报错信息: Traceback (most recent call last): File "train.py", line 232, in <module> main(args) File "train.py", line 227, in main has_dataset_json=args.has_dataset_json) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/core/train.py", line 138, in train logits_list = model(images) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(*inputs, kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, *kwargs) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 200, in forward x, x_downsample = self.backbone(x) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(inputs, kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(*inputs, kwargs) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 670, in forward out = self.forward_features(x) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 662, in forward_features x = layer(x) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(*inputs, *kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(inputs, kwargs) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 481, in forward x = blk(x) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 930, in call return self._dygraph_call_func(*inputs, *kwargs) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func outputs = self.forward(inputs, *kwargs) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 329, in forward assert L == H W, "input feature has wrong size" AssertionError: input feature has wrong size

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

marshall-dteach commented 1 year ago

还有你怎么知道这个是我复现的?哈哈

SurvivorNo1 commented 1 year ago

哈哈哈,谢谢回复。我在pullrequest里找的,主要是去PaddleSeg官方去问的话,太慢了,估计最后也是把问题分给你,想着直接问作者来得快一些,

SurvivorNo1 commented 1 year ago

您好,但是直接修改以后还是无法使用。

/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg
2023-04-15 21:08:38 [INFO]  
------------Environment Information-------------
platform: Linux-4.15.0-140-generic-x86_64-with-debian-stretch-sid
Python: 3.7.4 (default, Aug 13 2019, 20:35:49) [GCC 7.3.0]
Paddle compiled with cuda: True
NVCC: Build cuda_11.2.r11.2/compiler.29618528_0
cudnn: 8.2
GPUs used: 1
CUDA_VISIBLE_DEVICES: None
GPU: ['GPU 0: Tesla V100-SXM2-32GB']
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~16.04) 7.5.0
PaddlePaddle: 2.3.2
------------------------------------------------
2023-04-15 21:08:38 [INFO]  
---------------Config Information---------------
batch_size: 8
data_root: /tmp/data
export:
  inference_helper:
    type: InferenceHelper2D
  transforms:
  - size:
    - 1
    - 512
    - 512
    type: Resize3D
iters: 100000
loss:
  coef:
  - 1
  types:
  - coef:
    - 0.4
    - 0.6
    losses:
    - type: CrossEntropyLoss
    - type: DiceLoss
    type: MixedLoss
lr_scheduler:
  end_lr: 0
  learning_rate: 0.1
  power: 0.9
  type: PolynomialDecay
model:
  backbone:
    type: SwinTransformer_tinier_patch4_window7_224
  num_classes: 16
  type: SwinUNet
optimizer:
  momentum: 0.9
  type: sgd
  weight_decay: 0.0001
test_dataset:
  dataset_json_path: data/abdomen/abdomen_raw/dataset.json
  dataset_root: abdomen/abdomen_phase0
  mode: val
  num_classes: 16
  result_dir: abdomen/abdomen_phase1
  transforms:
  - size:
    - 1
    - 512
    - 512
    type: Resize3D
  type: Abdomen
train_dataset:
  dataset_root: abdomen/abdomen_phase0
  mode: train
  num_classes: 16
  result_dir: abdomen/abdomen_phase1
  transforms:
  - size:
    - 1
    - 512
    - 512
    type: Resize3D
  type: Abdomen
val_dataset:
  dataset_json_path: data/abdomen/abdomen_raw/dataset.json
  dataset_root: abdomen/abdomen_phase0
  mode: val
  num_classes: 16
  result_dir: abdomen/abdomen_phase1
  transforms:
  - size:
    - 1
    - 512
    - 512
    type: Resize3D
  type: Abdomen
------------------------------------------------
W0415 21:08:38.152426 10417 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0415 21:08:38.152477 10417 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.
Traceback (most recent call last):
  File "train.py", line 232, in <module>
    main(args)
  File "train.py", line 209, in main
    cfg.model,
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/cvlibs/config.py", line 320, in model
    self._model = self._load_object(model_cfg)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/cvlibs/config.py", line 425, in _load_object
    return component(**params)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 131, in __init__
    if (i_layer < self.num_layers - 1) else None)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 336, in __init__
    norm_layer=norm_layer) for i in range(depth)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 336, in <listcomp>
    norm_layer=norm_layer) for i in range(depth)
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 314, in __init__
    img_mask, self.window_size)  # nW, window_size, window_size, 1
  File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 78, in window_partition
    [B, H // window_size, window_size, W // window_size, window_size, C])
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/manipulation.py", line 2139, in reshape
    return paddle.fluid.layers.reshape(x=x, shape=shape, name=name)
  File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/nn.py", line 6373, in reshape
    out, _ = _C_ops.reshape2(x, None, 'shape', shape)
ValueError: (InvalidArgument) The 'shape' in ReshapeOp is invalid. The input tensor X'size must be equal to the capacity of 'shape'. But received X's shape = [1, 32, 32, 1], X's size = 1024, 'shape' is [1, 4, 7, 4, 7, 1], the capacity of 'shape' is 784.
  [Hint: Expected capacity == in_size, but received capacity:784 != in_size:1024.] (at /paddle/paddle/fluid/operators/reshape_op.cc:204)
  [operator < reshape2 > error]

直接修改img_size

    def __init__(self,
                 backbone,
                 img_size=512,
                 patch_size=4,
                 in_chans=3,
                 num_classes=1000,
                 embed_dim=96,
                 depths=[2, 2, 2, 2],
                 depths_decoder=[1, 2, 2, 2],
                 num_heads=[3, 6, 12, 24],
                 window_size=7,
                 mlp_ratio=4.,
                 qkv_bias=True,
                 qk_scale=None,
                 drop_rate=0.,
                 attn_drop_rate=0.,
                 drop_path_rate=0.1,
                 norm_layer=nn.LayerNorm,
                 patch_norm=True,
                 pretrained=None,
                 final_upsample=True,
                 **kwargs):
        super().__init__()
marshall-dteach commented 1 year ago

32//7不能整除,建议调整patch_size

------------------ 原始邮件 ------------------ 发件人: "marshall-dteach/SwinUNet" @.>; 发送时间: 2023年4月15日(星期六) 晚上9:13 @.>; @.**@.>; 主题: Re: [marshall-dteach/SwinUNet] SwinUnet可以使用[1,512,512]训练吗? (Issue #2)

您好,但是直接修改以后还是无法使用。 /home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg 2023-04-15 21:08:38 [INFO] ------------Environment Information------------- platform: Linux-4.15.0-140-generic-x86_64-with-debian-stretch-sid Python: 3.7.4 (default, Aug 13 2019, 20:35:49) [GCC 7.3.0] Paddle compiled with cuda: True NVCC: Build cuda_11.2.r11.2/compiler.29618528_0 cudnn: 8.2 GPUs used: 1 CUDA_VISIBLE_DEVICES: None GPU: ['GPU 0: Tesla V100-SXM2-32GB'] GCC: gcc (Ubuntu 7.5.0-3ubuntu1~16.04) 7.5.0 PaddlePaddle: 2.3.2 ------------------------------------------------ 2023-04-15 21:08:38 [INFO] ---------------Config Information--------------- batch_size: 8 data_root: /tmp/data export: inference_helper: type: InferenceHelper2D transforms: - size: - 1 - 512 - 512 type: Resize3D iters: 100000 loss: coef: - 1 types: - coef: - 0.4 - 0.6 losses: - type: CrossEntropyLoss - type: DiceLoss type: MixedLoss lr_scheduler: end_lr: 0 learning_rate: 0.1 power: 0.9 type: PolynomialDecay model: backbone: type: SwinTransformer_tinier_patch4_window7_224 num_classes: 16 type: SwinUNet optimizer: momentum: 0.9 type: sgd weight_decay: 0.0001 test_dataset: dataset_json_path: data/abdomen/abdomen_raw/dataset.json dataset_root: abdomen/abdomen_phase0 mode: val num_classes: 16 result_dir: abdomen/abdomen_phase1 transforms: - size: - 1 - 512 - 512 type: Resize3D type: Abdomen train_dataset: dataset_root: abdomen/abdomen_phase0 mode: train num_classes: 16 result_dir: abdomen/abdomen_phase1 transforms: - size: - 1 - 512 - 512 type: Resize3D type: Abdomen val_dataset: dataset_json_path: data/abdomen/abdomen_raw/dataset.json dataset_root: abdomen/abdomen_phase0 mode: val num_classes: 16 result_dir: abdomen/abdomen_phase1 transforms: - size: - 1 - 512 - 512 type: Resize3D type: Abdomen ------------------------------------------------ W0415 21:08:38.152426 10417 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2 W0415 21:08:38.152477 10417 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2. Traceback (most recent call last): File "train.py", line 232, in <module> main(args) File "train.py", line 209, in main cfg.model, File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/cvlibs/config.py", line 320, in model self._model = self._load_object(model_cfg) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/cvlibs/config.py", line 425, in _load_object return component(**params) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 131, in init if (i_layer < self.num_layers - 1) else None) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 336, in init norm_layer=norm_layer) for i in range(depth) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/swinunet.py", line 336, in <listcomp> norm_layer=norm_layer) for i in range(depth) File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 314, in init img_mask, self.window_size) # nW, window_size, window_size, 1 File "/home/aistudio/PaddleSeg-release-2.7/contrib/MedicalSeg/medicalseg/models/backbones/swin_transformer.py", line 78, in window_partition [B, H // window_size, window_size, W // window_size, windowsize, C]) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/manipulation.py", line 2139, in reshape return paddle.fluid.layers.reshape(x=x, shape=shape, name=name) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/nn.py", line 6373, in reshape out, = _C_ops.reshape2(x, None, 'shape', shape) ValueError: (InvalidArgument) The 'shape' in ReshapeOp is invalid. The input tensor X'size must be equal to the capacity of 'shape'. But received X's shape = [1, 32, 32, 1], X's size = 1024, 'shape' is [1, 4, 7, 4, 7, 1], the capacity of 'shape' is 784. [Hint: Expected capacity == in_size, but received capacity:784 != in_size:1024.] (at /paddle/paddle/fluid/operators/reshape_op.cc:204) [operator < reshape2 > error]

直接修改img_size def init(self, backbone, img_size=512, patch_size=4, in_chans=3, num_classes=1000, embed_dim=96, depths=[2, 2, 2, 2], depths_decoder=[1, 2, 2, 2], num_heads=[3, 6, 12, 24], window_size=7, mlp_ratio=4., qkv_bias=True, qk_scale=None, drop_rate=0., attn_drop_rate=0., drop_path_rate=0.1, norm_layer=nn.LayerNorm, patch_norm=True, pretrained=None, final_upsample=True, **kwargs): super().init()

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

SurvivorNo1 commented 1 year ago

抱歉再次打扰你, img_size,window_size,patch_size需要满足什么关系才能正常运行呢?

LebronMX-A commented 10 months ago

我改了一下内个patchsize就可以了

dumm-kd commented 10 months ago

我改了一下内个patchsize就可以了

可以问一下你改的好多吗,我试了几个都不行

congcongwy51 commented 3 months ago

请问您调节好了嘛,我想调节乘256尺寸的,应该怎么调节