raphaelreme / tps

Implementation of ThinPlateSpline
MIT License
20 stars 4 forks source link

Newbie Question #5

Closed JoSchmaltz closed 1 month ago

JoSchmaltz commented 2 months ago

I want to use this to warp 2d vector images (svg). I believe I understand the process as described for scikit here, but why is the numpy array for the control point X_c in the getting started code a 3D array and the target ndarray 2D? I expected both control and target to have the same dimension? Apologies, certainly a newbie here - and I dort even know if this is the place to ask.

raphaelreme commented 2 months ago

Hi !

It was just to show an example where the source space dimension (d_s) is different than the target space dimension (d_t), but you can look at the examples inside the example folder, you will see a 1d and 2d code, plus one for image warping.

In your case you will have d_s = d_t = 2. You can first generate a random transformation (calling fit on random control points, as done in the example for image warping), then you need to apply it to any element of your svg.

But this is not that easily done depending on the number of features you wanna support (i'm not so much familiar with the format, but if i'm not mistaken it can be quite complexe with some nested images & svg). But i guess that as a start, you can quite easily warp any polygons by warping the 2d positions of each vertex.

Let me know if this helps ;)

JoSchmaltz commented 1 month ago

That's great, thanks, I should have figured this out myself. I have managed to recreate something very similar to this (taken from here) with vectors or more specifically a geopandas dataframe instead of an image as in the examples.

Screenshot 2024-09-10 214513 With that I achieve my main objective which is to warp a OSM map, However, I am wondering (and this is probably more a algorithmic question than one for this repository) if there is a parameter that defined how "rigid" the structure is. I understand the control points will move to the target points with the alpha parameter defining how "exact". I also understand that points outside the control points will move subject to their distance to all control->target vectors. But how far will these points move? Is there a parameter that defines the how strong the reduction of "pull" from the c->t vectors is with increasing distance from these vectors? Sorry for my un mathematical lingo, but i am looking at this more with artistic intentions and little mathematical background. Thank you!

raphaelreme commented 1 month ago

Great to hear !

Sadly I'm no expert on the mathematical background neither, so I may not be aware of some other solutions that could already exist to control this "rigidity". This being said, here are some infos and tricks that may help you reach what you want:

  1. alpha should more be seen as a control of the smoothness of the solution. It allows to relax the constraints to get a more simple solution (meaning that for an increased smoothness of the deformation, the control points are allowed not to be warped perfectly). I didn't check, and I may be wrong, but i think that the higher the regularization alpha, the more the deformation tends to be linear and thus rigid.
  2. Another way to improve the resulting deformation is to better sample the control points. The estimated deformation is much less "precise" at location where no control point is found in the neighborhood. This is only an elastic interpolation method, so it is unable to predict a correct deformation outside of the control points domain and between control points it will predict a smooth elastic interpolation. Therefore I would advise to have at least control points in the corners (and borders ?) of your image, and also at least one for each deformation inside the image that you want to model.

Hope this helps!

JoSchmaltz commented 1 month ago

For what it's worth: a parameter to control the rigidity of the defornatiin seems to exist in the TPS concept:

The smoothing variant, correspondingly, uses a tuning parameter λ to control the rigidity of the deformation, balancing the aforementioned criterion with the measure of goodness of fit, thus minimizing.

I suspect this parameters exists somewhere in the code of this library. I will attempt to expose it such that it can be controlled easily.

raphaelreme commented 1 month ago

Yes this $\lambda$ in wikipedia is the parameter $\alpha$ in the code ;)

It allows you to control the regularization <=> rigidity (cf my previous message (1), it seems to be confirmed by what wikipedia says)

raphaelreme commented 1 month ago

I'm closing this issue. Do not hesitate to reopen it if needed ;)