训练实例分割模型报错

ChaoII commented 2 months ago

环境

操作系统：windows11
cpu：intel i7-12700
gpu: RTX4060TI
cuda 11.8

paddle 3.0.0b1

问题

在训练实例分割模型时报错，实例分割模型为paddlex/configs/instance_segmentation/Mask-RT-DETR-L.yaml，报错信息如下：


======================= Modified FLAGS detected =======================
FLAGS(name='FLAGS_win_cuda_bin_dir', current_value='E:\\develop\\minconda3\\envs\\paddle\\lib\\site-packages\\paddle\\..\\nvidia', default_value='')
=======================================================================
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
[07/17 15:14:19] ppdet.data.source.coco INFO: Load [79 samples valid, 0 samples invalid] in file D:\PaddleX\dataset\instance_seg_coco_examples\annotations/instance_train.json.
W0717 15:14:19.576007 37164 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.9, Driver API Version: 12.5, Runtime API Version: 11.8
W0717 15:14:19.577008 37164 gpu_resources.cc:164] device: 0, cuDNN Version: 8.9.
[07/17 15:14:20] ppdet.utils.checkpoint INFO: ['fc.bias', 'fc.weight', 'last_conv.weight'] in pretrained weight is not used in the model, and its will not be loaded
[07/17 15:14:20] ppdet.utils.checkpoint INFO: Finish loading model weights: C:\Users\AC/.cache/paddle/weights\PPHGNetV2_L_ssld_pretrained.pdparams
Traceback (most recent call last):
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\tools\train.py", line 213, in <module>
main()
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\tools\train.py", line 209, in main
run(FLAGS, cfg)
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\tools\train.py", line 162, in run
trainer.train(FLAGS.eval)
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\ppdet\engine\trainer.py", line 601, in train
outputs = model(data)
File "E:\develop\minconda3\envs\paddle\lib\site-packages\paddle\nn\layer\layers.py", line 1426, in __call__
return self.forward(*inputs, **kwargs)
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\ppdet\modeling\architectures\meta_arch.py", line 60, in forward
out = self.get_loss()
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\ppdet\modeling\architectures\detr.py", line 115, in get_loss
return self._forward()
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\ppdet\modeling\architectures\detr.py", line 89, in _forward
out_transformer = self.transformer(body_feats, pad_mask, self.inputs)
File "E:\develop\minconda3\envs\paddle\lib\site-packages\paddle\nn\layer\layers.py", line 1426, in __call__
return self.forward(*inputs, **kwargs)
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\ppdet\modeling\transformers\mask_rtdetr_transformer.py", line 326, in forward
get_denoising_training_group(gt_meta,
File "D:\PaddleX\paddlex\repo_manager\repos\PaddleDetection\ppdet\modeling\transformers\utils.py", line 169, in get_denoising_training_group
chosen_idx = paddle.nonzero(mask * pad_gt_mask).squeeze(-1)
TypeError: (InvalidType) Type promotion only support calculations between floating-point numbers and between complex and real numbers. But got different data type x: bool, y: float32. (at ..\paddle/phi/common/type_promotion.h:164)

Traceback (most recent call last): File "D:\PaddleX\paddlex\utils\result_saver.py", line 30, in wrap result = func(self, *args, kwargs) File "D:\PaddleX\paddlex\engine.py", line 42, in run trainer.train() File "D:\PaddleX\paddlex\modules\base\trainer\trainer.py", line 61, in train train_result = self.pdx_model.train(self.get_train_kwargs()) File "D:\PaddleX\paddlex\repo_apis\PaddleDetection_api\instance_seg\model.py", line 107, in train return self.runner.train( File "D:\PaddleX\paddlex\repo_apis\PaddleDetection_api\instance_seg\runner.py", line 54, in train return self.run_cmd( File "D:\PaddleX\paddlex\repo_apis\base\runner.py", line 359, in run_cmd raise CalledProcessError( paddlex.utils.errors.others.CalledProcessError: Command ['E:\develop\minconda3\envs\paddle\python.exe', 'tools/train.py', '--eval', '--config', 'C:\Users\AC\.paddlex\tmphaou86ix\instancesegmodel_Mask-RT-DETR-L.yml', '--use_vdl', 'True', '--vdl_log_dir', 'D:\PaddleX\output'] returned non-zero exit status 1.

cuicheng01 commented 2 months ago

您好，需要保证完全按照教程文档走哈~

ChaoII commented 2 months ago

你好，我是按照文档来的，安装是按照paddlex --install 来的，paddle版本 3.0.0b1

4 实例分割任务

4.1 数据校验

python main.py -c paddlex/configs/instance_segmentation/Mask-RT-DETR-L.yaml ^
    -o Global.mode=check_dataset ^
    -o Global.dataset_dir=./dataset/instance_seg_coco_examples

4.2 模型训练

python main.py -c paddlex/configs/instance_segmentation/Mask-RT-DETR-L.yaml ^
    -o Global.mode=train ^
    -o Global.dataset_dir=./dataset/instance_seg_coco_examples ^
    -o Global.device=gpu

4.3 模型评估

python main.py -c paddlex/configs/instance_segmentation/Mask-RT-DETR-L.yaml ^
    -o Global.mode=evaluate ^
    -o Global.dataset_dir=./dataset/instance_seg_coco_examples ^
    -o Global.device=gpu

4.4 模型预测

python main.py -c paddlex/configs/instance_segmentation/Mask-RT-DETR-L.yaml ^
    -o Global.mode=predict ^
    -o Predict.model_dir="./output/best_model/model" ^
    -o Predict.input_path="./dataset/instance_seg_coco_examples/images/beautiful-girl-smiling-da-128881.jpeg"

在4.1数据集校验通过执行训练后报上述错误。

mierzejk commented 1 month ago

@ChaoII Check out this RT-DETR commit 104afd3. Looks like a simple one line fix in the ppdet\modeling\transformers\utils.py file.

PaddlePaddle / PaddleX