search

facebookresearch / PoseDiffusion

[ICCV 2023] PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment
Other
718 stars 42 forks source link

Out of memory #21

Closed zhuhu00 closed 1 year ago

zhuhu00 commented 1 year ago

Hi, Jianyuan, I tried to retrain in co3dv2, but there was an “out of memory” error in only one or two categories. Is there any way to solve it? my device is RTX 3090 with 24GB memory.

image

jytime commented 1 year ago

Hi, you can make the value of max_images smaller. Sorry I should make it clearer. max_images actually takes the role of batch size in our code, because we use a dynamic batch size. In the code below, you can see n_seqs = self.max_images // n_per_seq. https://github.com/facebookresearch/PoseDiffusion/blob/8ce281338d46561f8a82b6f3a7675ad1712fc12f/pose_diffusion/util/train_util.py#L27-L59