Open yichao96 opened 4 years ago
Hi~ When I finetune VQA task, there is an error "Segmentation fault (core dumped)" The reason that my memory is 128G which is not enough , Could you give me some suggestion?
hallo! I tried to reproduce the result using 4*32G node, and it works. I think this is not caused by memory capacity which usually raise OOM problem.
The problem was solved, according to the issue (https://discuss.pytorch.org/t/segmentation-fault/23489).
You have to:
conda install pytorch==1.7.1 torchvision==0.8.2 cudatoolkit=10.1 -c pytorch
And if you followed the installation guide (https://github.com/microsoft/Oscar/blob/master/INSTALL.md), you also need to:
cd Oscar
or cd apex
rm -rf build/
python setup.py install --cuda_ext --cpp_ext
Oscar: python setup.py build develop
Hi~ When I finetune VQA task, there is an error "Segmentation fault (core dumped)" The reason that my memory is 128G which is not enough , Could you give me some suggestion?