OpenRobotLab / EmbodiedScan

[CVPR 2024] EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
https://tai-wang.github.io/embodiedscan/
Apache License 2.0
452 stars 34 forks source link

[Bug] the code take a long time to start running #78

Open Fan123456-c opened 2 hours ago

Fan123456-c commented 2 hours ago

Prerequisite

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

/

Reproduces the problem - code sample

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch \ --nproc_per_node=8 --master_port=29523 \ tools/train.py configs/grounding/mv-grounding_8xb12_embodiedscan-vg-9dof-full.py \ --work-dir=work_dirs/new-mv-gd --launcher="pytorch"

Reproduces the problem - command or script

"Why does the code take a long time to start running after I launch it? What is the reason for this, and is there any solution?"

Reproduces the problem - error message

It took a long time : 09/23 03:32:13 - mmengine - WARNING - Failed to search registry with scope "embodiedscan" in the "loop" registry tree. As a workaround, the current "loop" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "embodiedscan" is a correct scope, or whether the registry is initialized. [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 234014/234014, 14313.0 task/s, elapsed: 16s, ETA: 0s ] 0/234014, elapsed: 0s, ETA::

Additional information

No response

Fan123456-c commented 2 hours ago

It takes about 5 minutes from the time I launch the code until it starts running. What could be the reason for this delay, and is there any solution to reduce the startup time