search

jeanfeydy / geomloss

Geometric loss functions between point clouds, images and volumes
MIT License
599 stars 60 forks source link

Has ImagesLoss ever been finished? Or is it still a WIP? #65

Open kaselby opened 2 years ago

kaselby commented 2 years ago

In previous issues I saw mention of an implementation of this focused on images or segmentation maps. This would be extremely helpful for a project I am currently working on, but I don't see ImagesLoss in the code anywhere. The previous comments said it might be finished/added by early 2020 - am I just missing it, or was this never finished?

If it is still unfinished, would you be able to post the unfinished implementation so I can try to work with it? However unfinished it is, I'm sure it will certainly be better than whatever I try to implement from scratch myself.

jeanfeydy commented 2 years ago

Hi @krylea,

Thanks for your interest in the library! The proper integration of ImagesLoss is still WIP: unfortunately, I had to prioritize other works during the Covid years to secure a permanent academic job, which delayed the development for several years... But I am back to active coding now (in the ot_api branch) and will finally release all these features this summer. Until then, you may be interested by the geomloss/sinkhorn_images.py file that provides the "unofficial" implementation that you are looking for :-)

Best regards, And good luck for your project, Jean

pietz commented 2 years ago

When you say "images" you're referring to distributions in pixel space such as segmentation masks or heatmaps, correct? Because that's also what I'm looking for at the moment.

Thanks for your hard work!

jeanfeydy commented 2 years ago

Hi @pietz,

Yes indeed: density maps on 1D/2D/3D grids. I agree that this is a misleading terminology, since manipulating density is quite different to manipulating intensities (= what we tend to think about when we talk about natural images). From a mathematical perspective, the difference is in the impact of dilations/contractions on the domain: density values get scaled by the inverse of the local scaling factor (the Jacobian) to keep a constant sum on the domain, while intensities stay constant locally.

Optimal transport is all about densities - and so is GeomLoss for the time being. Extending these tools to intensities would be very useful, but I am not aware of a fully satisfying and tractable solution in the literature as of 2022.

Best regards, Jean

Perian-Yan commented 1 year ago

Hi, thank you for your great work!

If ImageLoss is still WIP, is there a workaround to use SampleLoss for density maps?