search

WangYueFt / dgcnn

MIT License
1.62k stars 420 forks source link

Question about seg in S3DIS #43

Open MrCrazyCrab opened 4 years ago

MrCrazyCrab commented 4 years ago

@WangYueFt Thanks for your Excellent work. I read the code about sem_seg, i meet a problem " ops['pointclouds_phs'][1]: current_data[start_idx_1:end_idx_1, :, :], IndexError: list index out of range" i am wondering about the ops, why is it necessary that the data is feeded twice, and how can i soleve the by changing the code? feed_dict = {ops['pointclouds_phs'][0]: current_data[start_idx_0:end_idx_0, :, :], ops['pointclouds_phs'][1]: current_data[start_idx_1:end_idx_1, :, :], ops['labels_phs'][0]: current_label[start_idx_0:end_idx_0], ops['labels_phs'][1]: current_label[start_idx_1:end_idx_1], ops['is_training_phs'][0]: is_training, ops['is_training_phs'][1]: is_training}

Looking forward to your reply!

jediofgever commented 4 years ago

I met same problem, I just removed 

ops['labels_phs'][1]: current_label[start_idx_1:end_idx_1],
ops['is_training_phs'][1]: is_training}

Bu I am not sure if it is right way to solve this. @ChenXie93 do you know what are the 9 values in .h5 files ? I assume the first 3 are x y z but I have no information about last 6

MrCrazyCrab commented 4 years ago

@jediofgever i guess ops['labels_phs'][1] is set for the training with two GPUs, so if you intent to use only one GPU, you can remove the code. Because i don't have a chance to train, so i'm not sure about if is right. As for the .h5 files, you can refer to the article "PointNet Deep Learning on Point Sets for 3D Classification and Segmentation", the first 3 are xys, the next 3 are RGB and the last is xys which are normolized.

jediofgever commented 4 years ago

Thanks for your info, I also don't have 2 gpus, are you trying to train network in your own data ?

MrCrazyCrab commented 4 years ago

@jediofgever , yeah . I change the code for 1 gpu training, and it didn't work. Anaconda does not report code errors， but the data cann't be feeded into the gpu. i cann't figure out till now. if you make it, may you share your experience?

jediofgever commented 4 years ago

@ChenXie93 , in my case I am trying to use my own dataset, when I removed those 2 lines I don't encounter that error anymore related to gpu, but I have problems to make placeholders to be compatible with my data type; for example the original input_place holder is as follows ; 

pointcloud_pl= (batch_size, num_points, 9)
label_pl= (batch_size, num_points)

but my point cloud does not have 9 values , only got x y z, so changed 9 => 3, but I still have error in label_pl part.

I think if you are trying to use your own data your problem also could be related to data compatibility to input placeholder

MrCrazyCrab commented 4 years ago

@jediofgever The first way is you can make up your data with gray value and the information of normalized xyz . Another way is chage the code just as you do, but i think you should pay attention to the construction of the cnn net(which you can see the code in model.py) rather than just change the 9 => 3. i incline to the first method for i don't think you are familiar with the entire code now.

jediofgever commented 4 years ago

@ChenXie93, in fact I don't have much experience in CNN and not familiar to codebase to modify architecture... thanks for your suggestions, I probably will do a training just with x, y, z values and labels. I will share results here for your reference, and after that I will also try adding up gray values and normalized xyz, and lastly what does normalized means here ? and how did you get these values ?

MrCrazyCrab commented 4 years ago

@jediofgever

> "Each point is represented by a 9-dim vector of XYZ, RGB and normalized location as to the room (from 0 to 1)." Cited from "PointNet Deep Learning on Point Sets for 3D Classification and Segmentation"

and you can refer some normalization methods to choose the bets one for your own data. As for the gray value, you are free to make up or replace RGB with other features . It would be ok if you process the data with the same approach, and you should know that all the time.

jediofgever commented 4 years ago

I understand the meaning of normalization now, it is just to compress xyz values between (0,1) with respect to some reference, appreciate your help much @ChenXie93

jediofgever commented 4 years ago

Hi @ChenXie93, I was able to train an object and network can perform quite ok despite my dataset being small size. Have you got it working yet I can probably share my experience so far, if you are still trying

MrCrazyCrab commented 4 years ago

@jediofgever , no, i still cannot train the dataset, could you share you experince on how to set the environment?

jediofgever commented 4 years ago

okay so briefly;

Data generation

After I get approximately 8192 .ply files, I squeeze these .ply files to 4 .h5 files for example the train0.h5 will have following keys and properties; 

data (2048,4096,9) 
label (2048,4096)

the script to generate .h5 training files is as follows

import h5py
import os, os.path
import numpy as np
from plyfile import PlyData, PlyElement

# directory that includes .ply files .ply, 1.ply ......... 8192.ply
DIR = '/home/atas/pcl_dataset/'

for m  in range(0,4):

    data = np.zeros((2048, 4096, 9), dtype = np.float32)
    label = np.zeros((2048,  4096),dtype = np.uint8)

    f = h5py.File('data'+str(m)+'.h5', 'w')

    for i in range(0, 2048):
        plydata = PlyData.read(DIR + str(m*2048+i) + ".ply")

        for j in range(0, 4096):
                    ## NORMALIZE POINT  CLOUD 
                    MAX_X, MAX_, MAX_Z = 4,4,4
                    MIN_X, MIN_Y, MIN_Z = -4,-4,-4

            data[i, j] = [plydata['vertex']['x'][j], plydata['vertex']['y'][j], plydata['vertex']['z'][j], plydata['vertex']['red'][j]/255.0, plydata['vertex']['green'][j]/255.0, 
                                  plydata['vertex']['blue'][j]/255.0,(plydata['vertex']['x'][j]-MIN_X)/8.0,(plydata['vertex']['y'][j]-MIN_Y)/8.0,(plydata['vertex']['z'][j]-MIN_Z)/8.0]

                      # The label information is embedded into rgb values, so if the rgb values are all 0 that mens the point is NONobject
                    if(plydata['vertex']['blue'][j] > 0 or plydata['vertex']['red'][j] > 0 or plydata['vertex']['green'][j] > 0 ):
                        label[i,j] = 1
                        #print("found object point")
                    else:
                        label[i,j] = 0

    f.create_dataset('data', data = data)
    f.create_dataset('label', data = label)

After you generate this training .h5 files, put them under dgcnn/ tensorflow/sem_seg/data and add a .txt(for example all_files.txt) file under this directory which includes path to your .h5 file like this;

data/data_train0.h5

Training

Simply I use the following to start training under sem_seg direcotry

python3 train.py --log_dir log1

in train.py , make sure ALL_FILES points to your .txt file which includes path to .h5 files you can see my branch which only have sem_seg https://github.com/jediofgever/dgcnn you can see last two commits that i changed to adapt code for myself

jediofgever commented 4 years ago

@ChenXie93 I am not sure at what point you are having problem, is that an code error ?

MrCrazyCrab commented 4 years ago

Thanks for your share, i have no problem in pre-processing my own data. In fact, the code itself dosen't has a error. However, when i run the code, the spyder stoped with the kernel restarted, and I found that the data were not feeded into the net. May be i need pay more attention to the code. Thank you so much for tell me the details, i 'm curious about one thing that whether you process your data through PCL? i don't know much about PCL function and i just random sample my data into 4096, so why do you need the PCL funtion to sample the data, is there any benifits to do it, more accuracy? Sorry to ask your so many questions, i'm just eager to find a way to solve my seg problem.