Custom dataset training

[SmitSheth]

Hi @LMescheder , I was trying to train using my own point cloud dataset (in .npy format). What is the format of the file points.npz which we need to create for our own dataset. Do we have to concatenate all the point clouds data in it? Between I am using onet.yaml file as the config file.

[itszash]

@SmitSheth how did you created point cloud dataset? i could really use some info on that

[jnyjxn]

@itszash @SmitSheth I'm not affiliated with this work, but I've found that there's a script called occupancy_networks/scripts/sample_mesh.py.

It turns an OFF mesh file into the points.npz and model.binvox files needed.

The code in the master branch needs a bit of tweaking to get it to work - specifically, the im2mesh/utils/binvox_rw.py and scripts/sample_mesh.py files. You can debug yourself or have a look at the changes I've made here.

Then you can run it like this (from occupancy_networks/scripts): python sample_mesh.py <path_containing_existing_off_file> --voxels_folder <output_folder> --points_folder <output_folder>

If your data is not in mesh form already then this won't work. If your mesh is in another format (PLY, OBJ, etc.) then you'll need to write a script to convert them - OFF is a very simple format fortunately. You'll also need your mesh to be watertight.

[LMescheder]

Thanks @jnyjxn for your explanation! Check out the build.sh script which contains all steps to build the ShapeNet dataset and which you can adapt to your own dataset. It calls the simple sample_mesh.py script which you should look at to learn how the data is stored.

[itszash]

@jnyjxn @LMescheder so wha i have to do first to train my data? converting my images data to mesh .off file?

[itszash]

@LMescheder @jnyjxn i have data of different image in jpg format what the next thing to do for training?

[jnyjxn]

@itszash This is a supervised machine learning model, so with every example (e.g. a set of images of the same object, as in your case) the algorithm needs to see the "correct answer" while training, which in the case of OccupancyNet seems to be a pointcloud and a voxel grid representing the 3D shape of the object.

Unfortunately if you don't have this 3D representation of the "correct answer" (you'll generally see it referred to as 'ground truth') then you'll need to find either an unsupervised technique (difficult) or a non-machine learning technique (for example, you can look up 'Structure-from-motion', 'Projective Reconstruction', etc.).

Then you can either generate a full training set (i.e. a dataset containing not just your jpg files but also your ground-truth OFF files) to train a supervised machine learning algorithm such as OccupancyNet, or you can avoid the need for machine learning altogether.

Hope that helps!

[itszash]

@jnyjxn yeah and my question is how can i give my data a 3d representation which is really important

[jnyjxn]

To extend my earlier answer somewhat:

If you don't have this 3D representation of the "correct answer" (you'll generally see it referred to as 'ground truth') then you'll need to find either an unsupervised technique (difficult) or a non-machine learning technique (for example, you can look up 'Structure-from-motion', 'Projective Reconstruction', etc.) and generate your 3D representation yourself.

If you don't already have a 3D representation and have no way to produce it, you probably won't be able to use machine learning to do reconstruction.

[ALI7861111]

@itszash @SmitSheth I'm not affiliated with this work, but I've found that there's a script called occupancy_networks/scripts/sample_mesh.py.

It turns an OFF mesh file into the points.npz and model.binvox files needed.

The code in the master branch needs a bit of tweaking to get it to work - specifically, the im2mesh/utils/binvox_rw.py and scripts/sample_mesh.py files. You can debug yourself or have a look at the changes I've made here.

Then you can run it like this (from occupancy_networks/scripts): python sample_mesh.py <path_containing_existing_off_file> --voxels_folder <output_folder> --points_folder <output_folder>

If your data is not in mesh form already then this won't work. If your mesh is in another format (PLY, OBJ, etc.) then you'll need to write a script to convert them - OFF is a very simple format fortunately. You'll also need your mesh to be watertight.

@itszash @SmitSheth I'm not affiliated with this work, but I've found that there's a script called occupancy_networks/scripts/sample_mesh.py.

It turns an OFF mesh file into the points.npz and model.binvox files needed.

The code in the master branch needs a bit of tweaking to get it to work - specifically, the im2mesh/utils/binvox_rw.py and scripts/sample_mesh.py files. You can debug yourself or have a look at the changes I've made here.

Then you can run it like this (from occupancy_networks/scripts): python sample_mesh.py <path_containing_existing_off_file> --voxels_folder <output_folder> --points_folder <output_folder>

If your data is not in mesh form already then this won't work. If your mesh is in another format (PLY, OBJ, etc.) then you'll need to write a script to convert them - OFF is a very simple format fortunately. You'll also need your mesh to be watertight.

Can you please tell the python shell script to run binvox_rw.py file and producing relevant output

[csyhping]

@itszash @SmitSheth I'm not affiliated with this work, but I've found that there's a script called occupancy_networks/scripts/sample_mesh.py.

It turns an OFF mesh file into the points.npz and model.binvox files needed.

The code in the master branch needs a bit of tweaking to get it to work - specifically, the im2mesh/utils/binvox_rw.py and scripts/sample_mesh.py files. You can debug yourself or have a look at the changes I've made here.

Then you can run it like this (from occupancy_networks/scripts): python sample_mesh.py <path_containing_existing_off_file> --voxels_folder <output_folder> --points_folder <output_folder>

If your data is not in mesh form already then this won't work. If your mesh is in another format (PLY, OBJ, etc.) then you'll need to write a script to convert them - OFF is a very simple format fortunately. You'll also need your mesh to be watertight.

Hi @jnyjxn , may I ask what is the model.binvox? Is it also generated by sample_mesh.py? If I wanna train with pointcloud, what should it be in the training .npz data?

[csyhping]

Hi @LMescheder , could you please give a description on the format of training data for pointcloud and what is inside the .npz files ? Thanks!