open-mmlab / mmrotate

OpenMMLab Rotated Object Detection Toolbox and Benchmark
https://mmrotate.readthedocs.io/en/latest/
Apache License 2.0
1.88k stars 558 forks source link

[Bug] MMDataParallel loads very slowly #1069

Open 1shenhui opened 1 month ago

1shenhui commented 1 month ago

Prerequisite

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

master branch https://github.com/open-mmlab/mmrotate

Environment

/hdc/miniconda3/envs/LSKNet/lib/python3.8/site-packages/mmcv/init.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details. warnings.warn( fatal: not a git repository (or any of the parent directories): .git sys.platform: linux Python: 3.8.18 | packaged by conda-forge | (default, Oct 10 2023, 15:44:36) [GCC 12.3.0] CUDA available: True GPU 0,1,2,3: NVIDIA A40-48Q CUDA_HOME: None GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0 PyTorch: 1.13.1+cu117 PyTorch compiling details: PyTorch built with:

TorchVision: 0.14.1+cu117 OpenCV: 4.8.0 MMCV: 1.7.1 MMCV Compiler: GCC 9.3 MMCV CUDA Compiler: 11.6 MMRotate: 0.3.4+

Reproduces the problem - code sample

no

Reproduces the problem - command or script

CUDA_VISIBLE_DEVICES=0 python /home/shenhui/semi-objetc-detection/SOOD-main/train.py /home/shenhui/semi-objetc-detection/SOOD-main/configs/ssad_fcos/dense_teacher_fcos_dota15_10per.py --work-dir /hdc/workspace_sh/debug

Reproduces the problem - error message

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.neck.fpn_convs.4.conv.weight - torch.Size([256, 256, 3, 3]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.neck.fpn_convs.4.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.0.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.0.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.0.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.1.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.1.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.1.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.2.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.2.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.2.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.3.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.3.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.cls_convs.3.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.0.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.0.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.0.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.1.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.1.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.1.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.2.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.2.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.2.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.3.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.3.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO - teacher.bbox_head.reg_convs.3.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_cls.weight - torch.Size([16, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_cls.bias - torch.Size([16]): NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_reg.weight - torch.Size([4, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_reg.bias - torch.Size([4]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_centerness.weight - torch.Size([1, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_centerness.bias - torch.Size([1]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_angle.weight - torch.Size([1, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.conv_angle.bias - torch.Size([1]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.scales.0.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.scales.1.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.scales.2.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.scales.3.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.scales.4.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO - teacher.bbox_head.scale_angle.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.conv1.weight - torch.Size([64, 3, 7, 7]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.conv1.weight - torch.Size([64, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.downsample.0.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO - student.backbone.layer1.0.downsample.1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.0.downsample.1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.1.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer1.2.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.conv1.weight - torch.Size([128, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO - student.backbone.layer2.0.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.0.downsample.0.weight - torch.Size([512, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.0.downsample.1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.0.downsample.1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.1.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.2.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.3.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.3.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.3.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.3.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.3.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.3.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO - student.backbone.layer2.3.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer2.3.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer2.3.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.conv1.weight - torch.Size([256, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.downsample.0.weight - torch.Size([1024, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.downsample.1.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.0.downsample.1.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.1.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.2.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.2.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.2.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO - student.backbone.layer3.2.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.2.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.2.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.2.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.2.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.2.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.3.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.4.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.5.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.5.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.5.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.5.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.5.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO - student.backbone.layer3.5.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer3.5.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer3.5.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer3.5.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.conv1.weight - torch.Size([512, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.downsample.0.weight - torch.Size([2048, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.downsample.1.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.0.downsample.1.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.1.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.2.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.2.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO - student.backbone.layer4.2.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO - student.backbone.layer4.2.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO - student.backbone.layer4.2.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO - student.backbone.layer4.2.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO - student.backbone.layer4.2.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO - student.backbone.layer4.2.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO - student.backbone.layer4.2.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.lateral_convs.0.conv.weight - torch.Size([256, 512, 1, 1]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.lateral_convs.0.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.lateral_convs.1.conv.weight - torch.Size([256, 1024, 1, 1]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.lateral_convs.1.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.lateral_convs.2.conv.weight - torch.Size([256, 2048, 1, 1]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.lateral_convs.2.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.0.conv.weight - torch.Size([256, 256, 3, 3]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.0.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.1.conv.weight - torch.Size([256, 256, 3, 3]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.1.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.2.conv.weight - torch.Size([256, 256, 3, 3]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.2.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.3.conv.weight - torch.Size([256, 256, 3, 3]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.3.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.4.conv.weight - torch.Size([256, 256, 3, 3]): XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.neck.fpn_convs.4.conv.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.bbox_head.cls_convs.0.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.bbox_head.cls_convs.0.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.bbox_head.cls_convs.0.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO - student.bbox_head.cls_convs.1.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO - student.bbox_head.cls_convs.1.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.cls_convs.1.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.cls_convs.2.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.cls_convs.2.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.cls_convs.2.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.cls_convs.3.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.cls_convs.3.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.cls_convs.3.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.0.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.0.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.0.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.1.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.1.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.1.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.2.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.2.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.2.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.3.conv.weight - torch.Size([256, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.3.gn.weight - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.reg_convs.3.gn.bias - torch.Size([256]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_cls.weight - torch.Size([16, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_cls.bias - torch.Size([16]): NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_reg.weight - torch.Size([4, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_reg.bias - torch.Size([4]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_centerness.weight - torch.Size([1, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_centerness.bias - torch.Size([1]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_angle.weight - torch.Size([1, 256, 3, 3]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.conv_angle.bias - torch.Size([1]): NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO - student.bbox_head.scales.0.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO - student.bbox_head.scales.1.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO - student.bbox_head.scales.2.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO - student.bbox_head.scales.3.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO - student.bbox_head.scales.4.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO - student.bbox_head.scale_angle.scale - torch.Size([]): The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

fatal: not a git repository (or any of the parent directories): .git

It was stuck here for a long time for more than half an hour and I found out by debugging that it's stuck here model = MMDataParallel( model.cuda(cfg.gpu_ids[0]), device_ids=cfg.gpu_ids) and The rest of the training became very slow, Both the GPU and the CPU have 0 usage

Additional information

No response

Leoeeeeeeea commented 1 week ago

have you solved this ?