Data processing issues on S3DIS

[zivlzw]

We confirm that the environment and related libraries have been configured as required, but when preparing the S3DIS dataset, the pkl file could not be generated after the ply file was successfully generated and an error was reported as follows. Is it convenient for you to upload a copy of the processed S3DIS dataset?

Availabel Mem of each GPU: 124268
Traceback (most recent call last):File "main.  py", line 100, in <module>trainer.  train()
File "/root/ERDA-main/utils/trainer.py", line 66, in traing= GraphBuilder (config, verbose-self.verbose)
File "/root/ERDA-main/utils/tf_graph_builder.py", line 47, in__init-dataset= self.  get_dataset (config, verbose)
File "/root/ERDA-main/models/../datasets/tf_s3dis_dataset.py", line 112, in__init,lself.  load_subsampled_clouds(config.  first_subsampling_dl, verbose-verbose
File "/root/ERDA-main/models/.. /datasets/tf_s3dis dataset.py", line 650, in load subsanpled_clouds_alignedsub_xyz, sub_colors, sub_labels= grid_subsampling (xyz, colors, labels, subsampling_parameter)
File "/root/ERDA-main/ops/tf_ops.py", line 121, in grid_subsamplingreturn cpp_subsampling,compute(points, features-features, classes-labels, sanpleDl-sampleDl, verbose-verbose)
AttributeError: module ops.  cpp wrappers.  cpp subsampling, grid subsampling' has no attribute'compute

[zivlzw]

We confirm that the environment and related libraries have been configured as required, but when preparing the S3DIS dataset, the pkl file could not be generated after the ply file was successfully generated and an error was reported as follows. Is it convenient for you to upload a copy of the processed S3DIS dataset?

Availabel Mem of each GPU: 124268
Traceback (most recent call last):File "main.  py", line 100, in <module>trainer.  train()
File "/root/ERDA-main/utils/trainer.py", line 66, in traing= GraphBuilder (config, verbose-self.verbose)
File "/root/ERDA-main/utils/tf_graph_builder.py", line 47, in__init-dataset= self.  get_dataset (config, verbose)
File "/root/ERDA-main/models/../datasets/tf_s3dis_dataset.py", line 112, in__init,lself.  load_subsampled_clouds(config.  first_subsampling_dl, verbose-verbose
File "/root/ERDA-main/models/.. /datasets/tf_s3dis dataset.py", line 650, in load subsanpled_clouds_alignedsub_xyz, sub_colors, sub_labels= grid_subsampling (xyz, colors, labels, subsampling_parameter)
File "/root/ERDA-main/ops/tf_ops.py", line 121, in grid_subsamplingreturn cpp_subsampling,compute(points, features-features, classes-labels, sanpleDl-sampleDl, verbose-verbose)
AttributeError: module ops.  cpp wrappers.  cpp subsampling, grid subsampling' has no attribute'compute

We have solved this problem and it is the reason that the library does not compile well.

[LiyaoTang]

Hi,

Thanks for your interest. Glad that you have solved the problem.

Cheers, Liyao

[mooncakehub]

when I process the S3DIS,I met this error,did you meet

[LiyaoTang]

Hi,

Thanks for your interest.

It seems that you use the wrong files. We use the 'Aligned_Version'. Could you check that?

Best, Liyao

[mooncakehub]

than you for your reply. I check my dataset,it's definit ' Aligned_Version',but the method of processed dataset is for ' Stanford3dDataset_v1.2'. so I check the code. I found that default version is 'aligned',but when processed,it became 'full'.I don't know how did this happen,so I modified the version 'aligned'

[mooncakehub]

How much video memory is needed to process the s3dis dataset, I use 4 2080TI gpus, the computer will automatically shut down and abort the process

[LiyaoTang]

Hi,

It's probably the problem with the config that you use, please check the version in the config and if it changes during the code execution.

It's also glad to see that you made some progress. To process the S3DIS for the first time, I would recommend having around 64GB CPU mem.

[mooncakehub]

Hi， I am really grateful for your reply. I used 'free- h' to check CPU mem, the CPU mem is enough.so Do you mean that the GPU has 64G of video mem

[mooncakehub]

[LiyaoTang]

Hi,

It seems your hardware should be fine to run the code. Could you check the log and try to provide more details (output printing / trace-back if any ) regarding the process shut-down?