Issues with custom data

[piseabhijeet]

Hi @riccardomarin

Amazing work,

I tried to use the PointNet version of your network on my custom dataset but i get a huge error:

My dataset - a custom dataset with 4 classes 2k vertices and 45 Eigen values.

What could be wrong?

Thanks

[riccardomarin]

Hi @piseabhijeet , thank you for your message.

Without further information this is not trivial to guess where is the problem. Few questions: 1) are you using the pretrained network + your data, or are you training the network from scratch on your data? 2) What are exactly your data? (i.e., which class of object, number of models) are they on the same scale and spatially aligned? 3) Which LBO version are you using? 4) Can you report the training loss curves?

[piseabhijeet]

Hi @riccardomarin

That was the quickest response i ever got :) thank you

To answer your questions:

1. No, i am training the network from scratch
2. The dataset comprises of 4 classes of utensils - .obj files with vertices and meshes - yes, they are in same scale and are not spatially aligned - not in [0,1] or [-1,1] range. The obj files are read and saved using igl library _igl.read_trianglemesh()
3. I am using linear FEM from https://github.com/nipy/mindboggle/blob/master/mindboggle/shapes/laplace_beltrami.py
4. I guess the summaries were missing in the code. Is there a simpler way to add them and visualize ?

Thanks

[riccardomarin]

Thank you for your reply!

Are your objects watertight shapes (i.e., they are all single connected component)? Can you send to me some examples of the four classes (even images are fine)?

Yes, in the code I haven't reported the summaries. I think the easiest way is to save at each epoch the loss in a numpy vector and write it in a .npy at the end of the training (at the moment I think the code is printing it so it should be not difficult to store the losses in a variable).

Also, I think a good debugging would be check if: a) different spectra are sent to different points in the latent space (i.e., the Pi map is not collapsed) b) different input shapes are sent to different points in the latent spaces (i.e., the encoder succeed to produce a meaningful latent space) c) if the twos are fine, you can check if different points of the latent space are sent to different spectra (i.e., check if the Rho network is not collapsed).

I think this would help to identify which part of the network has failed.

[piseabhijeet]

Hi @riccardomarin

Unfortunately, i cant share the images or obj files because of some non-disclosure clauses.

However, I added TF summaries to your code and the N2 loss is oscillating a lot, guess that's the main culprit here:

Also, i noticed, you are not normalizing the eigen values and have used them in the same range [1+, 350+], is there a reason for the same?

About your points: a) Yes i have visualized the latent space and they are meaningful I will have to check points b and c.

Thank you

[piseabhijeet]

Hi @riccardomarin ,

I also noticed the following: the losses for the original code are without 'reduce_mean', so i added reduce_mean to get a scalar for the summary and obtained incorrect results:

, however, when i kept the original loss for N1, N2 and train for around 25k steps with a reduced batch size, i am getting some results which are in 10-15% error range.

[riccardomarin]

Hi @piseabhijeet ,

indeed the N2 loss (that is the one from the latent space to the spectrum) seems unstable. I think you could try to augment its weight. But also I think that inspecting the spectrum distribution would help; if they are too different across the shapes it may results that it easier send them into the latent space, and harder the inverse direction. If your object classes are significantly different, I can expect something like that.

An approach that we are investigating in the following works is encoding the spectrum not in the absolute values, but as the discrete derivative. For example, given a spectrum with values: [1,5,7,10] we use the encoding [1,4,2,3] This seems more stable because the spectrum is asymptotically linear, hence such values should be within a certain range. In some case, we also normalize the shape for the areas.

Concerning the reduce_mean, I am not sure how this could impact the training, also because N2 and N1 already present the "tf.keras.losses.mean_squared_error". I do not know if it is related to some differences in the TF versions, or some randomicity in the training that my code does not handle.

[piseabhijeet]

Hi @riccardomarin

Thank you for the response.

This is how my original spectra looks like (20 samples with 5 examples of each class): ![Uploading image.png…]()

I am taking first 45 values (excluding 0 and scaling others by the first non zero eigen value). What do you think about this spectra?

[piseabhijeet]

[riccardomarin]

@piseabhijeet thank you for the image. Yes, the spectra look a bit sparse in their slopes, and in their absolute values. Maybe an attempt would be normalizing the shapes for their area (s.t. each shape has Area == 1), and don't normalize the spectra for the first eigenvalue.

[piseabhijeet]

@riccardomarin thank you for the prompt response. Will definitely try that out :)

[piseabhijeet]

Hi @riccardomarin

The main cause of the behavior was the dataset was not axis aligned completely. I could overtrain on a single class to get a better decoded shape now. :)