search

lidq92 / MDTVSFA

[official] Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training (IJCV 2021)
MIT License
83 stars 16 forks source link

Why not use multiple gpu? #1

Closed ciwei123 closed 3 years ago

ciwei123 commented 3 years ago

@lidq92 ,Hi, thanks for your sharing，and I found that the code is running on a single gpu ，why not run on multiple GPUs? I can successfully run the CNNfeature.py on a single gpu，but when I use multiprocessing , I meet the error:

THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=50 error=3 : initialization error Process Process-8: Traceback (most recent call last): File "/root/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/root/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/MDTVSFA-main/CNNfeatures_mp.py", line 147, in run features = get_features(current_data['video'], frame_batch_size, model, device) File "/MDTVSFA-main/CNNfeatures_mp.py", line 121, in get_features output1 = torch.Tensor().to(device) File "/root/anaconda3/lib/python3.6/site-packages/torch/cuda/init.py", line 193, in _lazy_init torch._C._cuda_init() RuntimeError: cuda runtime error (3) : initialization error at /pytorch/aten/src/THC/THCGeneral.cpp:50

lidq92 commented 3 years ago

@ciwei123 You can simply use nn.DataParallel with multiple GPUs to enlarge the true batch size. And you can refer to lines 587-590 in lidq92/WaDIQaM/main.py for a simple illustration.

Running on a single GPU is just OK for people who use only one GPU. If the GPU memory is not enough, one can even set frame_batch_size=1.

ciwei123 commented 3 years ago

@lidq92 Thanks for your reply . And there is another question. I can not open the "FORM" needed to download the dataset. Could you share LIVE-Qualcomm dataset according to Baidudisk. Thank you very much!

lidq92 commented 3 years ago

@ciwei123 See https://github.com/lidq92/VSFA/issues/18#issuecomment-654080057 and https://github.com/lidq92/VSFA/issues/30#issuecomment-742584641

@lidq92 Thanks for your reply . And there is another question. I can not open the "FORM" needed to download the dataset. Could you share LIVE-Qualcomm dataset according to Baidudisk. Thank you very much!