ValueError: need at least one array to stack

[Cemm23333]

Dear Dr.Robert: I ran s3dis_visualization.py and I met ValueError.How to fix it?

[Cemm23333]

Dear Dr.Robert: I downloaded the dataset Stanford3dDataset_v1.2.zip and put it under "/s3disfused/".But all the files of the generated raw datatset are .txt files and I can't find expected image directory structure in /DeepViewAgg/torch_points3d/datasets/segmentation/multimodal/s3dis.py. Does Stanford3dDataset_v1.2.zip refer to the 2D-3D-Semantics ?

[drprojects]

Hi @Cemm23333, thanks for documenting this issue :raised_hands: !

I have to apologize, I seem to have omitted the download automation for the multimodal S3DIS dataset :grimacing:

For now, you need do manually download the images from 2D-3D-Semantics and place them in your dataroot directory (next to the dataroot/raw directory) as indicated here.

If I remember correctly, there might be a difference in room alignments between the 3D data you get from 2D-3D-Semantics and the 3D data from S3DIS. Just to be safe and avoid redownloading what you already have, make sure you backup somwhere the 3D data you already got from S3DIS.

My code takes care of correcting some incorrect camera and room orientations, but unfortunately I can't remember if I applied this placing the 3D data from 2D-3D-Semantics or from S3DIS (though I tend to lean towards the former) under dataroot/root/raw.

Again, I apologize for this, I thought I had fully automated the 2D+3D S3DIS download pipeline.

Please let me know how that goes and I will try to automate it in the future !

[drprojects]

Hi @Cemm23333 have you manually downloaded the data to solve this issue ?

[ruomingzhai]

Hi @drprojects . I have manually downlad the no_xyz s3dis dataset. The S3DIS download automation also bothers me because in the default _download function in torch_geometric/data/dataset.py. ` def _download(self): if files_exist(self.raw_paths): # pragma: no cover return

    makedirs(self.raw_dir)
    self.download()`

The _self.rawpaths for S3DIS refers to

But the actual folders are as followed:

It inevitably leads to redownload function. How can I skip this download function? I am confused a little bit. Hope you can help me.

[drprojects]

Hi @ruomingzhai, I am currently out of office and kind of off-the-grid until mid september.

I will try to help you in the meantime. As you saw, pytorch geometric datasets triggers self.download() if any of the files/folders in self.raw_paths is absent. Self.raw_paths are actually computed based on file/folder names declared in self.raw_file_names (see here ).

There is indeed an error here for my S3DIS dataset using images. As stated above, I have not coded the multimodal S3DIS dataset download automation yet, I am sorry for the inconvenience. Right now the self.raw_paths correspond to where the raw 3D data is expected (in your case for no_xyz you still need the 3D data to have acess to point labels and positions to build 2D-3D mappings).

The 3D data should be downloaded from the S3DIS dataset here and unzipped in the root/raw/Area_{1,2,3,4,5,6}

The 2D data (images and poses) which you seem to have already downloaded, must be downloaded from here and unzipped following the structure indicated here.

If the files are properly located, the self.download function should be skipped. So far you could not move further because you did not have the 3d data where it should be.

Next the self.process should work. In particular, it is important to note that there are camera pose errors in 2D-3D Semantics and room orientation errors in S3DIS. It is one of the reasons why I recommend you to download 3D data from one dataset and 2D data from another. I managed to fix those errors and the self.process should apply necessary changes. I know this part is a little hacky and am sorry I have not properly automated yet, I know it can be a bit frustrating.

Hope this helps. If you are still stuck, let me know, I will reply after sept 15th.

[ruomingzhai]

Hi @ruomingzhai, I am currently out of office and kind of off-the-grid until mid september.

I will try to help you in the meantime. As you saw, pytorch geometric datasets triggers self.download() if any of the files/folders in self.raw_paths is absent. Self.raw_paths are actually computed based on file/folder names declared in self.raw_file_names (see here ).

There is indeed an error here for my S3DIS dataset using images. As stated above, I have not coded the multimodal S3DIS dataset download automation yet, I am sorry for the inconvenience. Right now the self.raw_paths correspond to where the raw 3D data is expected (in your case for no_xyz you still need the 3D data to have acess to point labels and positions to build 2D-3D mappings).

The 3D data should be downloaded from the S3DIS dataset here and unzipped in the root/raw/Area_{1,2,3,4,5,6}

The 2D data (images and poses) which you seem to have already downloaded, must be downloaded from here and unzipped following the structure indicated here.

If the files are properly located, the self.download function should be skipped. So far you could not move further because you did not have the 3d data where it should be.

Next the self.process should work. In particular, it is important to note that there are camera pose errors in 2D-3D Semantics and room orientation errors in S3DIS. It is one of the reasons why I recommend you to download 3D data from one dataset and 2D data from another. I managed to fix those errors and the self.process should apply necessary changes. I know this part is a little hacky and am sorry I have not properly automated yet, I know it can be a bit frustrating.

Hope this helps. If you are still stuck, let me know, I will reply after sept 15th.

Thank you for the tips. It works right now with the right folder path for downloaded dataset!

[drprojects]

Hi @Cemm23333 and @ruomingzhai, I looked into S3DIS 2D data download automation.

Glad to hear you got things working @ruomingzhai.

I just updated the repo: you should now be able to automatically download 3D data from S3DIS and 2D data from 2D-3D-Semantics. In case the automatic download fails (which happens more often than we would like, due to massive file sizes...), the user should see a message giving instructions for manual download.

Unfortunately, I was not able to fully test it because the 2D-3D-Semantics Google Drive seems to be having issues, but I have no power on that !

Please let me know if this works for you @Cemm23333 or @ruomingzhai. Leaving this issue open in the meantime.

[drprojects]

Hi @ruomingzhai @Cemm23333 can you please let me know if the above message solved your issue ?

[ruomingzhai]

Hi @ruomingzhai @Cemm23333 can you please let me know if the above message solved your issue ?

Thanks a lot! It works as I put the manually downloaded dataset in the right file folder.