Jumpat / SegAnyGAussians

The official implementation of SAGA (Segment Any 3D GAussians)
Apache License 2.0
472 stars 36 forks source link

cuda out of memory #58

Open zw20130717 opened 1 month ago

zw20130717 commented 1 month ago

when i ran extract_features.py using the command in github page, i was informed cuda out of memory. I'm new to coding and python project. After searching the Internet, I changed the batch_size and downsample and still was unable to solve the problem. The imgs I used was 1074*544. Befor SAGA I had successfully run the 3DGS project and received no error information. Can anyone offer any advice which i will be very appreciated (except for change of hardware) 2024-05-23 17-04-49 的屏幕截图

Jumpat commented 1 month ago

Hi you may need 24GB GPU memory for training SAGA. SAGA attaches additional features to the original GS model and the training requires more memory

zw20130717 commented 1 month ago

Hi you may need 24GB GPU memory for training SAGA. SAGA attaches additional features to the original GS model and the training requires more memory

omg 24GB. I presume this is way beyond the efforts of code changing and image editting to solve this cuda-out-of-memory problem. Anyway, thanks for your reply. I may have to borrow someone else's computer to run this project.

Jumpat commented 1 month ago

You're welcome. We are using a 3090. There may be some method to cut down GPU consumption. For example you can move the sam_masks to cpu. Or you can downsample the masks for supervision, or sample some masks (do not use all masks in an iteration of training). You can check it by yourself.

Luca-Wiehe commented 1 month ago

In the v2-version of the project, I have the same problem despite having 24GB of GPU memory. I trained SAM masks using the --downsample 4 flag.

Since you have the same GPU limitations that we have, which downsampling factor did you use for SAM masks? Did you do anything else to reduce GPU requirements? I am training using a 2000 image scene from the Replica dataset that you reported in your paper.

Edit: Solved this by downsampling the number of images from my dataset.