search

allenai / objaverse-xl

🪐 Objaverse-XL is a Universe of 10M+ 3D Objects. Contains API Scripts for Downloading and Processing!
https://objaverse.allenai.org/
Apache License 2.0
704 stars 40 forks source link

What is the Objaverse-XL storage size (inodes & blocks) ? #12

Open hunoutl opened 11 months ago

hunoutl commented 11 months ago

Do you have some numbers on the Objaverse-XL size once downloaded ? (with 3D objects) ? What impacts does github repo have ? with zip/tar/tar.gz/files ?

mattdeitke commented 11 months ago

See https://huggingface.co/datasets/allenai/objaverse-xl/discussions/5

Not sure on the storage size for the objects themselves.

For rendering, 12 renders for each object (plus the metadata and npy files) is about 1.75MB. Multiply that by 10M and its about 17.5 TB.

Others are welcome to chime in if they download everything and store it all.

The scripts were made such that you don't have to store individual objects to be able to process all of it (i.e., see the callbacks).

ouceduxzk commented 1 month ago

i download the 800k objaverse 1.0, which takes 10TB storage. so the whole 10M objaverse-xl problably takes 100T