left/right inverses?

[cscherrer]

Just adding a note here (we don't have to dig in right now), that it would be great to also support left_inverse and right_inverse, since this comes up so often

[oschulz]

Good point, let's keep this issue as a reminder.

[sethaxen]

I agree, this would be useful to have. left_inverse and right_inverse could fall back to inverse, so that if a downstream user only needs to require that a function be right-invertible, then they would call right_inverse directly and support more functions.

[oschulz]

Also relates to #8, e.g. for square/sqrt.

[oschulz]

right-inverses could also be helpful components (building blocks) in pullback definitions, see example in JuliaObjects/Accessors.jl#46 (right_inverse(@optic(_.a) would be it's pullback, modulo ChainRulesCore boilerplate).

[ParadaCarleton]

@oschulz I see sqrt and square are defined as inverses, even though this isn't strictly correct; is the plan to allow partial inverses (e.g. trig/hyptrig) functions to be defined as inverse, or should I define a new function partial_inverse for this case? (We've come across it in TableTransforms.jl, where we need left inverses.)

[oschulz]

@ParadaCarleton , so far we haven't really had a use case for left/right inverses, but I guess now there is one with TableTransforms then. I think there's consensus that in principle we should add left/right inverses. Would you like to put a proposal together, maybe as a draft PR - just bare-bone initially, so we can discuss the API?

[aplavin]

Noticed interesting convergence, though arriving from quite a different direction.

In AccessorsExtra, I define the construct([T,] f1 => val1, f2 => val2, ...) function to create an object with the given values function. The result is an object of type T with f1(obj) == val1, f2(obj) == val2, ... .

In the general case, construct methods have to be defined for each set of functions that one wants to create an object from, such as

construct(Gaussian, area => 1, fwhm => 2)
construct(Complex, abs => 3, angle => 0.5)

However, invertible functions and basic types like (named)tuples are supported automatically. This means

julia> rinverse(f) = x -> construct(f => x)

is a right inverse for supported functions.

# just uses inverse() inside
julia> rinverse(log10)(3)
1000.0

# uses inverse() + creates namedtuple
julia> rinverse(@o log10(_.a))(3)
(a = 1000.0,)

I don't really have common usecases for right inverses to non-fully-invertible mathematical functions, so this rinverse only supports invertible functions + property or index accessors.

It can be seen from these examples that right_inverse is kinda "more general" than the full inverse and can naturally be applied to things like property accessors – not just mathematical functions.

[oschulz]

rinverse(f) = x -> construct(f => x)

Oh, this looks elegant @aplavin !

I tried running construct(log => 0.5) , but I get no method matching default_empty_obj(::typeof(log)) - how do I tell construct what to build?

[oschulz]

Works beautifully @aplavin ! :-)

[oschulz]

I've been thinking, if we offer left/right-inverses, should we allow room in the API and namespace for non-function inverses, and a more general package "InverseElements.jl" (or simply "Inverses.jl")?

Our current inverse(f) could mathematically (an inverse being in relation to a binary operation and a neutral element) be written as inverse_element(∘, identity, f). Such an API would then also allow for inverse_element(+, 0, 5) and so on. I don't think we need to pass the neutral element explicitly though, I think we can treat it as implied by the operation and the object.

So we could have a package InverseElements that defines inverse_element(op, object), it could depend on InverseFunctions and define inverse_element(::typeof(∘), f) = inverse(f).

But what about left_inverse and right_inverse? Should they be for functions only, then we'd also need left_inverse_element and right_inverse_element for the general case, or should we require passing op (so ∘ for functions) explicitly for left_inverse and right_inverse in general?

[oschulz]

Works beautifully not

Quite hard to parse as a non-native speaker – does it work or doesn't? :)

It does work - my bad, the perils of typing on the phone on a train. :-)

[oschulz]

Interesting thoughts about inverse_element, do you have any specific usecases in mind?

We would actually have at lease one nice internal use case: We could use expand inverse support for Base.Fix1 and Base.Fix2 and make it more elegant and less awkward dispatch-wise.

Let's say we do it all in "InverseFunctions.jl" (no extra package "Inverse[Elements].jl"). "InverseFunctions" can mean "functions the generate inverses" just as well as "inverses of functions", after all. Then we could have a generic implementation

inverse(ff::Base.Fix1) = Base.Fix1(ff.op, left_inverse(ff.op, ff.x))
inverse(ff::Base.Fix2) = Base.Fix2(ff.op, right_inverse(ff.op, ff.x))

And with left/right-inverses for specific binary operations

left_inverse(::typeof(-), x::Real) = x
right_inverse(::typeof(-), x::Real) = -x

we'd automatically get

julia> f = Base.Fix1(-, 5); inverse(f)(f(9))
9

julia> f = Base.Fix2(-, 5); inverse(f)(f(9))
9

We still might want to specialize inverse of Base.Fix1 and Base.Fix2 for certain operations (e.g. so that \ is used in case of inv to inverse matrix multiplication), but in general the above should work nicely, I think.

[oschulz]

*'s inverse would be / for performance/accuracy

Not necessarily, at least not for numbers - if the inverse is used more than once, then precomputing 1/x would result in much higher performance (since division is so much slower than multiplication), and I think accuracy wouldn't suffer (I think many ALUs compute division via multiplication with inverse anyway). For matrices it's a different story - precomputing the inverse might seem attractive if the inverse is used multiple times, but as far as I know using \ can be a lot more stable numerically (depending on the matrix) than multiplication with the inverse matrix.

Certainly wouldn't hurt to define left_inverse and right_inverse for all of +, -, * and /, and then only specialize Base.Fix... inverses where required, I'd say.

^'s inverse cannot be done with ^ at all

Unfortunately not - (b ^ e) ^ (1//x) does work even for negative b, but ^ is not associative.

So yes, we'd still need quite a few explicit Base.Fix... inverses, but I think it would still be elegant. And it would add left- and right-inverses in a way that extends the scope of the fashion in a natural and self-consistent fashion.

[MilesCranmer]

What’s the current right way to handle this? I’m looking to use something like this in SymbolicRegression. FWIW I like the inverse(::Base.Fix1/2) idea, which was the first thing I tried before hitting the error.

[oschulz]

What’s the current right way to handle this?

Left/right- inverses of a specific inverse case involving FixN? For left/right-inverses we need to add that to the API - I think there's consensus that we want to have it, but no one has done a PR with an API for it yet. :-)

[MilesCranmer]

Gotcha.

Just to check, is the following the right idea?

inverse(f::Base.Fix1{typeof(+)}) = Base.Fix2(-, f.x)

I'm starting out with an internal method that forwards to InverseFunctions to avoid piracy and since for my use-case it's useful to have non-exact inverses (like cos -> acos or even things like abs -> abs) here: https://github.com/MilesCranmer/SymbolicRegression.jl/blob/a7f207abd9518c6e9f9a59bc1d64a8c2e1de72da/src/InverseFunctions.jl

[MilesCranmer]

I guess there's some ambiguity here... not sure if there's a better way

inverse(f::Base.Fix1{typeof(+)}) = Base.Fix1(+, -f.x)

Some are quite nice though:

inverse(f::Base.Fix1{typeof(-)}) = f
inverse(f::Base.Fix1{typeof(/)}) = f

[MilesCranmer]

For posterity, here's what I put into SR.jl. Note that I label these as approx_inverse just to avoid pirating InverseFunctions.jl. And also for the fact that some may not have inverses for part of the domain (still useful for my stuff)

# (f.x + _) => (_ - f.x)
approx_inverse(f::Base.Fix1{typeof(+)}) = Base.Fix2(-, f.x)
# (_ + f.x) => (_ - f.x)
approx_inverse(f::Base.Fix2{typeof(+)}) = Base.Fix2(-, f.x)

# (f.x * _) => (_ / f.x)
approx_inverse(f::Base.Fix1{typeof(*)}) = Base.Fix2(/, f.x)
# (_ * f.x) => (_ / f.x)
approx_inverse(f::Base.Fix2{typeof(*)}) = Base.Fix2(/, f.x)

# (f.x - _) => (f.x - _)
approx_inverse(f::Base.Fix1{typeof(-)}) = f
# (_ - f.x) => (_ + f.x)
approx_inverse(f::Base.Fix2{typeof(-)}) = Base.Fix2(+, f.x)

# (f.x / _) => (f.x / _)
approx_inverse(f::Base.Fix1{typeof(/)}) = f
# (_ / f.x) => (_ * f.x)
approx_inverse(f::Base.Fix2{typeof(/)}) = Base.Fix2(*, f.x)

# (f.x ^ _) => log(f.x, _)
approx_inverse(f::Base.Fix1{typeof(^)}) = Base.Fix1(log, f.x)
# (_ ^ f.x) => _ ^ (1/f.x)
approx_inverse(f::Base.Fix2{typeof(^)}) = Base.Fix2(^, inv(f.x))

[MilesCranmer]

D’oh!! I need sleep…

[oschulz]

@MilesCranmer , if you don't specifically need FixN, you can also use AffineMaps, it has dedicated objects for Mul, Add, MulAdd and AddMul, with support for InverseFunctions and ChangesOfVariables. So it would be Add(b) instead of Base.Fix2(+, b).