search

ethnhe / raster_triangle

A simple renderer with z-buffer for synthesis data generating.
56 stars 21 forks source link

How much disk space does the file take up #12

Closed guess-who-ami closed 2 years ago

guess-who-ami commented 2 years ago

after this command python3 rgbd_renderer.py --cls ape --render_num 70000 I have generated > 100G files and haven't stopped

guess-who-ami commented 2 years ago

about 200G

LEONHWH commented 2 years ago

about 200G

Oh god, only 'ape' for 200G?

guess-who-ami commented 2 years ago

about 200G

Oh god, only 'ape' for 200G?

yes, it is. Is anything wrong?

LEONHWH commented 2 years ago

about 200G

Oh god, only 'ape' for 200G?

yes, it is. Is anything wrong?

I mean if I'm going to train pvn3d or FFB6D, whether I have to execute the code python3 fuse.py --cls & python3 rgbd_renderer.py --cls per object? In that case, to train 13 objects in the entire linemod dataset, I may have to prepare 13*200G of memory space....

guess-who-ami commented 2 years ago

you can generate one object dataset, and train the model for the object. Then generate the next dataset. In that way, you can only use 200G memory space.

ethnhe commented 2 years ago

There are several ways to save memory space:

  1. Randomly sample the pose and generate less than 70000 scenes. I think about 10000 scenes are enough to get a good performance.
  2. Save the generated RGB images to ".jpg", segmentation mask to ".png" using cv2.imwrite; convert the depth image to be in mm and uint16 data type and save it to ".png" using pypng; save the remaining info dictionary to pickle. That will save a lot of space compared to the full-pickle version as it has compression. I think it will take about 10G for the full 70000 scenes in this way.