search

lerogo / aaai24_itr_cusa

Source code of our AAAI 2024 paper "Cross-Modal and Uni-Modal Soft-Label Alignment for Image-Text Retrieval"
24 stars 3 forks source link

启动时候报错 #9

Open SleepBoomFat opened 3 weeks ago

SleepBoomFat commented 3 weeks ago

Creating model ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 0 (pid: 8278) of binary: /opt/conda/envs/t20240823/bin/python Traceback (most recent call last): File "/opt/conda/envs/t20240823/bin/torchrun", line 8, in sys.exit(main()) File "/opt/conda/envs/t20240823/lib/python3.9/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 346, in wrapper return f(*args, **kwargs) File "/opt/conda/envs/t20240823/lib/python3.9/site-packages/torch/distributed/run.py", line 794, in main run(args) File "/opt/conda/envs/t20240823/lib/python3.9/site-packages/torch/distributed/run.py", line 785, in run elastic_launch( File "/opt/conda/envs/t20240823/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 134, in call return launch_agent(self._config, self._entrypoint, list(args)) File "/opt/conda/envs/t20240823/lib/python3.9/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 请问一下这个问题，用的单卡训练。。

lerogo commented 3 weeks ago

启动命令是啥，什么卡，什么驱动版本

SleepBoomFat commented 3 weeks ago

torchrun --nnodes=1 --nproc_per_node=1 retrieval.py --config "./configs/test.yaml" Tesla-P40 CUDA12.4呢，是不是不能用这个卡跑呀

lerogo commented 3 weeks ago

看一下你的torch版本和cuda和你的p40卡配套与否。p40我没有跑过，我不知道

SleepBoomFat commented 3 weeks ago

谢谢大佬，我换成4090就可以跑了。还有个问题哈，triton好像目前仅支持linux，如果想在windows本地跑一下模型，该怎么处理呢