How to config to train on NVIDIA GeForce RTX 3090 Ti/PCIe/SSE2

[vtmjapandev]

Hi guys,

I want to train this model on(NVIDIA GeForce RTX 2080 Ti/PCIe/SSE2), RAM 16GB, How to config it. When I ran the default setting, the bellow error has occored.

    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

[taohan10200]

see https://github.com/taohan10200/IIM/issues/28#issue-1164642630

[vtmjapandev]

@taohan10200 I have set CUDA_LAUNCH_BLOCKING=1 but the same error has occurred.

[vtmjapandev]

@taohan10200 In the dataset setting file with NWPU, I have set the batch size to 1. But this batch size is good? __C_NWPU.TRAIN_BATCH_SIZE = 1 #imgs

[vtmjapandev]

@taohan10200 I have set batch_size from 12 -> 6. But when traint, bellow error has occurred.

python3.7/site-packages/torch/nn/functional.py:3704: UserWarning: nn.functional.upsample is deprecated. Use
nn.functional.interpolate instead.
  warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
./run_train.sh: line 6: 75217 Segmentation fault      (core dumped) python3.7 train.py

[taohan10200]

Based on my experience, this error occurs when some packages you pip install are not the right version. Please check each required package with the import xxx command to locate the error package.

[vtmjapandev]

@taohan10200 You can share with me your requrements.txt file.

The current I ran the project on Ubuntu 20.04 Python3.7 GTX 3090