Fail to recongnise equivalence of two forms of the sigmoid function

[oxinabox]

The following two equations should be equivalent (for using SymEngine; @vars z) exp(z)/(1+exp(z)) and 1/(1+exp(-z))

This is what my hand-math tells me, and it is listed on wikipedia.

But does not work

• exp(z)/(1+exp(z)) == 1/(1+exp(-z)) --> false
• expand(exp(z)/(1+exp(z))) == expand(1/(1+exp(-z))) --> false

Similar expression involving multiplication instread of division work -- if you call expand

• exp(-z)*(1+exp(z)) == 1 + exp(-z) --> false
• expand(exp(-z)*(1+exp(z))) == expand(1 + exp(-z))

I guess this could be broken into two issues.

1. divisions don't "expand" for correct equality
2. expansion should be called before equality is checked.

[isuruf]

I don't think expand should do this simplification. Following works.

julia> SymEngine.as_numer_denom(exp(z)/(1+exp(z))) == SymEngine.as_numer_denom(1/(1 + exp(-z)))
true

or

julia> num(exp(z)/(1+exp(z))) == num(1/(1 + exp(-z)))
true

julia> den(exp(z)/(1+exp(z))) == den(1/(1 + exp(-z)))
true

expansion should be called before equality is checked.

It's theoretically impossible to determine the mathematical equivalence of symbolic equations. So, for equality, we are using a well-defined equivalence, namely structural equivalence where the expression trees are compared.

Mathematical equivalence can be checked only using heuristics which SymPy does using a combination of expand, as_numer_denom, trigsimp, etc. SymEngine doesn't have a simplification routine yet, but hopefully we'll have one soon.

[oxinabox]

This seems to go a long way:

function basic_simplify(f)
    num,den = as_numer_denom(f)
    f2 = inv(expand(inv(num)*den)) # make it single line with cancellations done

    num2,den2 = as_numer_denom(f2) #make it back into a multiline
    num2/den2
end

test cases:

using Base.Test
@vars z
s1 = (1 + exp(-z))^(-1)
s2 = exp(z)/(1+exp(z))

@test basic_simplify(s1)==basic_simplify(s2)
@test basic_simplify(inv(s1))==basic_simplify(inv(s2))

n1 = subs(s1, z=>-z)
n2 = 1-s1
n3 = subs(s2, z=>-z)
n4 = 1-s2
@test basic_simplify(n1)==basic_simplify(n2)==basic_simplify(n3)==basic_simplify(n4)
@test basic_simplify(inv(n1))==basic_simplify(inv(n2))==basic_simplify(inv(n3))==basic_simplify(inv(n4))

as_numer_denom on its own fails at n3

[oxinabox]

I've continued to poke at this problem. I think maybe there is nothing to be done. The tools to fix this are just not exposed to julia, so I can't write my own simplify.

Using

function ugly_simplify(f)
    num,den = as_numer_denom(f)
    inv(expand(inv(num)*den))
end

function basic_simplify(f)
    num2,den2 = as_numer_denom(ugly_simplify(f))
    num2/den2
end

(a)≖(b) = ugly_simplify(a)==ugly_simplify(b)

This attempt works for some cases:

g1 = exp(-z1)/(1+exp(-z1))^2
g2 = inv(1 + exp(-z1)) * inv(1 + exp(z1))
g1 ≖ g2

But not for others, where it just won't force it to the same structure

@vars z1 z2
f1 = exp(-z1)/(1+exp(-z1)^2) + exp(-z2)/(1+exp(-z2)^2)
f2 = sum(inv.(1 + exp.([z1,z2])) .* inv.(1 + exp.(-[z1,z2])))
f1 ≖ f2 # Should be true, but is false

[isuruf]

Looks like as_numer_denom can be improved. https://github.com/symengine/symengine/issues/1327

[mikolajpp]

This is tangentially related, but still. Hope you won't take offence @isuruf :-)

@oxinabox I have been looking at symengine, but I have come the conclusion, that as far as julia goes, it is a wrong way to go. All basic structures should just be s-expressions, which julia directly exposes. I tried to do some cool stuff, like 2nd quantization formalism, but implementing anything like that is a huge pain in symengine. julia was specifically created to get rid of 2 language problem.

I have been toying with some basic symbolic library in pure julia, might try to get in better shape and post to github.

Meanwhile, thanks @isuruf for your efforts. Symengine.jl is definitely useful, but I would be catious with spending effort trying to make it into full-blown julia library. Without direct mapping to native julia typesystem, it will always feel very inflexible.

[ChrisRackauckas]

I have been toying with some basic symbolic library in pure julia, might try to get in better shape and post to github.

While I agree that type dispatch would make extending this stuff a lot easier, it's a little quick to say we should stop using this before there's a viable alternative. As it stands, if you want features then SymPy.jl is the best game in town and if you can live with a smaller feature set but need speed SymEngine.jl is the way to go. SymEngine/SymPy has the backing of a big development community which can be hard to match. However, please go ahead and develop a native Julia version and I'll definitely take a look.

[mikolajpp]

I am far from saying that. I have been using various libraries so far, recently have been trying out the Reduce.jl.

SymEngine.jl is actually very good for certain tasks, but personally I badly need something that is hackable: no julia user wants to hear that she would have to alter c++ code.

Over time I hope we could have some basic skeleton to which people would gradually add more and more clever algorithms.