search

salesforce / ULIP

BSD 3-Clause "New" or "Revised" License
437 stars 44 forks source link

What is the size of ShapeNet55 here? #32

Closed Jingnan-Jia closed 1 year ago

Jingnan-Jia commented 1 year ago

I am downloading the dataset ShapeNet55 from the link in ULIP readme file.

But one hour passes the dataset is still downloading. And I found that there is already over 150 GB now!

Can anyone tell me the total size of this dataset? If it is too big for my disk I will have to stop the downloading earlier.

Jingnan-Jia commented 1 year ago

Oh, I found the answer:

[140.9 GiB/ 1.1 TiB] 12% Done 38.5 MiB/s ETA 07:15:35

It is over 1 TiB ! It is too big for my workstation. I stopped the downloading.

Tycho-Xue commented 1 year ago

Hi @Jingnan-Jia , if it helps, you can refer to here https://github.com/salesforce/ULIP/issues/7#issue-1618575476. the current version contains some more types of images, maybe you can try to use regex for RGB images and depth maps only when downloading, and I will try to upload a new version that only contains images of RGB and depth maps, which should reduce some storage.

Tycho-Xue commented 1 year ago

@Jingnan-Jia , I've uploaded a "only_rgb_depth_images" folder, which is around 420GB, it contains all the images I used for the pre-training, which is much smaller than the previous full image folder.