Open cscherrer opened 2 years ago
Good point, let's keep this issue as a reminder.
I agree, this would be useful to have. left_inverse
and right_inverse
could fall back to inverse
, so that if a downstream user only needs to require that a function be right-invertible, then they would call right_inverse
directly and support more functions.
Also relates to #8, e.g. for square
/sqrt
.
right-inverses could also be helpful components (building blocks) in pullback definitions, see example in JuliaObjects/Accessors.jl#46 (right_inverse(@optic(_.a)
would be it's pullback, modulo ChainRulesCore boilerplate).
@oschulz I see sqrt
and square
are defined as inverses, even though this isn't strictly correct; is the plan to allow partial inverses (e.g. trig/hyptrig) functions to be defined as inverse
, or should I define a new function partial_inverse
for this case? (We've come across it in TableTransforms.jl, where we need left inverses.)
@ParadaCarleton , so far we haven't really had a use case for left/right inverses, but I guess now there is one with TableTransforms then. I think there's consensus that in principle we should add left/right inverses. Would you like to put a proposal together, maybe as a draft PR - just bare-bone initially, so we can discuss the API?
Noticed interesting convergence, though arriving from quite a different direction.
In AccessorsExtra
, I define the construct([T,] f1 => val1, f2 => val2, ...)
function to create an object with the given values function. The result is an object of type T
with f1(obj) == val1
, f2(obj) == val2
, ... .
In the general case, construct
methods have to be defined for each set of functions that one wants to create an object from, such as
construct(Gaussian, area => 1, fwhm => 2)
construct(Complex, abs => 3, angle => 0.5)
However, invertible functions and basic types like (named)tuples are supported automatically. This means
julia> rinverse(f) = x -> construct(f => x)
is a right inverse for supported functions.
# just uses inverse() inside
julia> rinverse(log10)(3)
1000.0
# uses inverse() + creates namedtuple
julia> rinverse(@o log10(_.a))(3)
(a = 1000.0,)
I don't really have common usecases for right inverses to non-fully-invertible mathematical functions, so this rinverse
only supports invertible functions + property or index accessors.
It can be seen from these examples that right_inverse
is kinda "more general" than the full inverse and can naturally be applied to things like property accessors – not just mathematical functions.
rinverse(f) = x -> construct(f => x)
Oh, this looks elegant @aplavin !
I tried running construct(log => 0.5)
, but I get no method matching default_empty_obj(::typeof(log))
- how do I tell construct what to build?
Works beautifully @aplavin ! :-)
I've been thinking, if we offer left/right-inverses, should we allow room in the API and namespace for non-function inverses, and a more general package "InverseElements.jl" (or simply "Inverses.jl")?
Our current inverse(f)
could mathematically (an inverse being in relation to a binary operation and a neutral element) be written as inverse_element(∘, identity, f)
. Such an API would then also allow for inverse_element(+, 0, 5)
and so on. I don't think we need to pass the neutral element explicitly though, I think we can treat it as implied by the operation and the object.
So we could have a package InverseElements
that defines inverse_element(op, object)
, it could depend on InverseFunctions
and define inverse_element(::typeof(∘), f) = inverse(f)
.
But what about left_inverse
and right_inverse
? Should they be for functions only, then we'd also need left_inverse_element
and right_inverse_element
for the general case, or should we require passing op
(so ∘
for functions) explicitly for left_inverse
and right_inverse
in general?
Works beautifully not
Quite hard to parse as a non-native speaker – does it work or doesn't? :)
It does work - my bad, the perils of typing on the phone on a train. :-)
Interesting thoughts about inverse_element, do you have any specific usecases in mind?
We would actually have at lease one nice internal use case: We could use expand inverse
support for Base.Fix1
and Base.Fix2
and make it more elegant and less awkward dispatch-wise.
Let's say we do it all in "InverseFunctions.jl" (no extra package "Inverse[Elements].jl"). "InverseFunctions" can mean "functions the generate inverses" just as well as "inverses of functions", after all. Then we could have a generic implementation
inverse(ff::Base.Fix1) = Base.Fix1(ff.op, left_inverse(ff.op, ff.x))
inverse(ff::Base.Fix2) = Base.Fix2(ff.op, right_inverse(ff.op, ff.x))
And with left/right-inverses for specific binary operations
left_inverse(::typeof(-), x::Real) = x
right_inverse(::typeof(-), x::Real) = -x
we'd automatically get
julia> f = Base.Fix1(-, 5); inverse(f)(f(9))
9
julia> f = Base.Fix2(-, 5); inverse(f)(f(9))
9
We still might want to specialize inverse
of Base.Fix1
and Base.Fix2
for certain operations (e.g. so that \
is used in case of inv
to inverse matrix multiplication), but in general the above should work nicely, I think.
*'s inverse would be / for performance/accuracy
Not necessarily, at least not for numbers - if the inverse is used more than once, then precomputing 1/x
would result in much higher performance (since division is so much slower than multiplication), and I think accuracy wouldn't suffer (I think many ALUs compute division via multiplication with inverse anyway). For matrices it's a different story - precomputing the inverse might seem attractive if the inverse is used multiple times, but as far as I know using \
can be a lot more stable numerically (depending on the matrix) than multiplication with the inverse matrix.
Certainly wouldn't hurt to define left_inverse
and right_inverse
for all of +
, -
, *
and /
, and then only specialize Base.Fix...
inverses where required, I'd say.
^'s inverse cannot be done with ^ at all
Unfortunately not - (b ^ e) ^ (1//x)
does work even for negative b, but ^
is not associative.
So yes, we'd still need quite a few explicit Base.Fix...
inverses, but I think it would still be elegant. And it would add left- and right-inverses in a way that extends the scope of the fashion in a natural and self-consistent fashion.
What’s the current right way to handle this? I’m looking to use something like this in SymbolicRegression. FWIW I like the inverse(::Base.Fix1/2)
idea, which was the first thing I tried before hitting the error.
What’s the current right way to handle this?
Left/right- inverses of a specific inverse case involving FixN
? For left/right-inverses we need to add that to the API - I think there's consensus that we want to have it, but no one has done a PR with an API for it yet. :-)
Gotcha.
Just to check, is the following the right idea?
inverse(f::Base.Fix1{typeof(+)}) = Base.Fix2(-, f.x)
I'm starting out with an internal method that forwards to InverseFunctions to avoid piracy and since for my use-case it's useful to have non-exact inverses (like cos -> acos
or even things like abs -> abs
) here: https://github.com/MilesCranmer/SymbolicRegression.jl/blob/a7f207abd9518c6e9f9a59bc1d64a8c2e1de72da/src/InverseFunctions.jl
I guess there's some ambiguity here... not sure if there's a better way
inverse(f::Base.Fix1{typeof(+)}) = Base.Fix1(+, -f.x)
Some are quite nice though:
inverse(f::Base.Fix1{typeof(-)}) = f
inverse(f::Base.Fix1{typeof(/)}) = f
For posterity, here's what I put into SR.jl. Note that I label these as approx_inverse
just to avoid pirating InverseFunctions.jl. And also for the fact that some may not have inverses for part of the domain (still useful for my stuff)
# (f.x + _) => (_ - f.x)
approx_inverse(f::Base.Fix1{typeof(+)}) = Base.Fix2(-, f.x)
# (_ + f.x) => (_ - f.x)
approx_inverse(f::Base.Fix2{typeof(+)}) = Base.Fix2(-, f.x)
# (f.x * _) => (_ / f.x)
approx_inverse(f::Base.Fix1{typeof(*)}) = Base.Fix2(/, f.x)
# (_ * f.x) => (_ / f.x)
approx_inverse(f::Base.Fix2{typeof(*)}) = Base.Fix2(/, f.x)
# (f.x - _) => (f.x - _)
approx_inverse(f::Base.Fix1{typeof(-)}) = f
# (_ - f.x) => (_ + f.x)
approx_inverse(f::Base.Fix2{typeof(-)}) = Base.Fix2(+, f.x)
# (f.x / _) => (f.x / _)
approx_inverse(f::Base.Fix1{typeof(/)}) = f
# (_ / f.x) => (_ * f.x)
approx_inverse(f::Base.Fix2{typeof(/)}) = Base.Fix2(*, f.x)
# (f.x ^ _) => log(f.x, _)
approx_inverse(f::Base.Fix1{typeof(^)}) = Base.Fix1(log, f.x)
# (_ ^ f.x) => _ ^ (1/f.x)
approx_inverse(f::Base.Fix2{typeof(^)}) = Base.Fix2(^, inv(f.x))
D’oh!! I need sleep…
@MilesCranmer , if you don't specifically need FixN
, you can also use AffineMaps, it has dedicated objects for Mul
, Add
, MulAdd
and AddMul
, with support for InverseFunctions and ChangesOfVariables. So it would be Add(b)
instead of Base.Fix2(+, b)
.
Just adding a note here (we don't have to dig in right now), that it would be great to also support
left_inverse
andright_inverse
, since this comes up so often