jeanfeydy
commented
2 years ago Hi @gabrielfougeron,
Thanks for your interest in the library!
To answer your questions:
-
The library supports float32 by default. It is by far the most relevant numerical type for optimal transport computations: it is well supported by both CPU and GPU and provides the necessary precision without being “overkill” like float64.
-
However, it does not really support float16 computations; I do not plan to add support for low-precision types in the near future. This is due to the fact that on current hardware, float16 computations only speed-up computations for a restricted set of operations which are not super relevant for optimal transport solvers. Implementing e.g. the Sinkhorn algorithm with matrix-vector multiplications with float16 numbers is even more unstable than in the standard float32 setting. LogSumExp reductions are crucial here, and I don’t think that they would be faster with float16 numbers. I may be wrong though, so I will try this in my next GeomLoss programming session.
-
Finally, current priority for me is to clean up the library, add dedicated support for 2D/3D grids and implement a mature API. Right now, I believe that run times are much less of an issue than the general “user experience” which has stagnated since 2019 ;-)
Best regards, Jean
gabrielfougeron
Hi,
It looks to me as though the different algorithms for computing OT losses in Geomloss do not have support for float16/float32 dtypes. Is this correct? Would it even make sense? (I'm thinking about the impact it might have on the stability of the algorithms) Are there any plans to add this in the future?
Best,