search

PRBonn / LiDiff

[CVPR'24] Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion
MIT License
221 stars 19 forks source link

train.py doesn't work #24

Closed naonao-wu closed 3 months ago

naonao-wu commented 4 months ago

Hello When I run python3 train.py on a terminal using a GPU, I get an error cuda out of memory, But when I increase the number of GPU and an error pytorch_lightning. Utilities. Exceptions. MisconfigurationException: you requested to check 0.001 of the 'val_dataloader' but 0.001 * 679 < 1. Please increase the 'limit_val_batches' flag. Try at least 'limit_val_batches=0.0014727540500736377', when I change the limit_val_batches value in train.py file according to the error message, And gave me an error cuda out of memory, has been infinite loop, how should I solve??

My environment is PyTorch 1.9.0, Python 3.8(ubuntu18.04) ,Cuda 11.1 , RTX3080.

To be sure I am running python3 map_from_scans. py - path Datasets/SemanticKITTI/dataset/sequences/run directly after the command python3 train. py.

nuneslu commented 4 months ago

Hi! You need to run python3 map_from_scans. py - path Datasets/SemanticKITTI/dataset/sequences/ to create the ground truth to be used during the training. Regarding the cuda out of memory, it seems that your GPU doesn't have enough memory to run the code. You can change some configs on configs/config.yaml to fit the model in your GPU, for example, decrease the batch_size to 1, or decrease max_range, or decrease num_points.

nuneslu commented 3 months ago

I will close this issue now. In case you have other questions, feel free to reopen it.