Convergence and results

[ethancohen123]

Hi, I am trying to use you package for doing some statistical shape analysis following your notebook. It works fine with your dataset with the bones , but with my data the mean do not converge ( I have tried to increase the maxIteration ) and the results are not good . Do you know what may cause that and how I can fixed it ? Thank you

[jdtuck]

So this depends on what your data looks like which leads me to a few questions:

1. Are they open or closed curves?
2. How many samples per curve?
3. How are you calling the fdacurve?
4. Can you share a picture of your data after you load into fdacurve, using the plot method?

[ethancohen123]

1.There are closed curves

2. 150 samples ( I can modify that as I take the contour points from a binary mask )
3. Like this : obj = fs.fdacurve(shapes,N=M,mode='C',scale=True) 4 . Attached

[jdtuck]

So can you plot a few of the curves, some don’t even look to be from the same shape, but there are so many overlapped. Also a first try, set the mode to ‘O’ and see if we get a solution.

[jdtuck]

Also can you show me what you mean by the ‘results are not good’

[jdtuck]

Also one more check, are you sure they are closed (i.e., the last point is the same as the first point). The reason I ask is that from the plot (could be plotting) it looks like some are not.

[ethancohen123]

Thanks , I checked and the curves are indeed not close; the first and last point are a little bit different Here is the some curves (10 ) plotted PS: The curves represent the contour points of embryos shape at same age and same phenotype About the results , here is what I get with 66 curves

• It reaches maxIter and the optimization for the mean do not converge ( I've tried for maxIter=50 and 100 but it is the same )
• The mean shape doesnt look like what we expect for the mean shape ; The middle of the shape is not supposed to be that thin looking at the original data
• The PC1 looks a bit weird : I am not sure but it seems that it the results should be more 'aligned ' as with the bones dataset in the example notebook Thank you for you time

[jdtuck]

Okay, please run it as open curves and let me know if you get a similar result.

[ethancohen123]

It is the results with open curves

[jdtuck]

Hmm, there is something weird in a few of your curves then. I would check for duplicate points, also you can use the resamplecurve function to resample them all to say 100 points and see if that changes anything. It is something in the data. Can you share it?

[ethancohen123]

Here is a zip file with the 66 shapes in txt files. There is 150 points per curve control_point.zip

[jdtuck]

So there is a bug, I am fixing now. The curves are supposed to be centered, in my example they are so close to the same its not a big deal, yours are not on centered at the origin. Will let you know once I release a fix. Working on it now. Sorry about that and thank you for asking your question

[jdtuck]

Have pushed the fix to the repo, have not released it to pip till I have done some more testing. I still don't get results on your data that make 100% sense. I have run your curves on the original MATLAB code and get the same result on open curves. There is something in your data that I am trying to figure out.

[jdtuck]

So after looking closer at your data you had curves sampled clockwise and others counterclockwise and the code assumes they are all in one direction. I am making a fix to fdacurve to check for this.

[ethancohen123]

Thank you very much for your time For the last point isnt it something that the framework should do ? I mean in the SRVF framework the registration problem is taken care implicitly during the optimization for the mean . So if the curves are registered , rotation shouldnt be removed right ? Unless the sampling problem is different from a rotation problem

[jdtuck]

So the SRVF handles rotation, scale, and reparameterization of the shapes. Rotation is the actual rotation of the shape in \mathbb{R}^n, where you would apply a rotation matrix. What I am talking about is not that. You had shapes that started from one starting point and traversed in sampling to the endpoint. You had other shapes that started at the endpoint and sampled to the starting point. Part of the SRVF is taking a derivative and these derivatives have different signs. The code assumed all curves in the set were sampled from one direction, especially when you choose open curves. For closed curves, it will align the seed point (starting), but still assumes you sampled in the same direction at least. It is an easy fix. Let me know how your results look at the latest commit.

[ethancohen123]

Okay thank you , it is much clearer Can I use it with pip or should I clone the git repo

[jdtuck]

Currently, you need to clone the repo, I have not released to pip or bumped the version yet

[jdtuck]

I got good results (what I think) on your data for open curves when I tested last night

[ethancohen123]

I have tested it , it is much better thank you !

[ethancohen123]

Also , if I turned them into closed curves , the bug is also fixed ? Because , in a next step, I want to align them with respect to a starting point that I will define . The starting point is a label point that I have for each curves corresponding to a biological feature in the embryos

[jdtuck]

Good to hear, and yes it is fixed for both open and closed.

[jdtuck]

I will tag a release here shortly