Example doesn't turn out well

[ChrisRackauckas]

The example was probably done on an old version of the code with different hyperparameters. These days... the result using the parameters from the example is really bad.

using TSne, MNIST

rescale(A, dim::Integer=1) = (A .- mean(A, dim)) ./ max.(std(A, dim), eps())

data, labels = traindata()
data = convert(Matrix{Float64}, data[:, 1:2500])'
# Normalize the data, this should be done if there are large scale differences in the dataset
X = rescale(data, 1)

Y = tsne(X, 2, 50, 1000, 20.0)

using Plots
pyplot()
scatter(Y[:,1], Y[:,2], groups=string.(labels[1:size(Y,1)]))
savefig("tsne.png")

[ChrisRackauckas]

Maybe this could have happened after @alyst 's distance change? I remember the package working months back.

[alyst]

@ChrisRackauckas Thanks for the report! Indeed it's because of the #18 changes to pca(): I didn't realize that eigfact() sorts eigvalues in ascending order. Should be fixed now.

[lejon]

Many thanks for the fix @alyst!

[ChrisRackauckas]

Could this get a patch release?

[ChrisRackauckas]

Yup it did, thanks!

[alyst]

JuliaLang/METADATA.jl#14214

[ChrisRackauckas]

I am still getting something that doesn't look so good:

using TSne, MNIST

rescale(A, dim::Integer=1) = (A .- mean(A, dim)) ./ max.(std(A, dim), eps())

data, labels = traindata()
data = convert(Matrix{Float64}, data[:, 1:2500])'
# Normalize the data, this should be done if there are large scale differences in the dataset
X = rescale(data, 1)

Y = tsne(X, 2, 50, 1000, 20.0)

using Plots; plotly()
scatter(Y[:,1],Y[:,2],color=Int.(labels[1:size(Y,1)]))

[alyst]

@ChrisRackauckas Is it something new or #21 didn't fix it for you?

[ChrisRackauckas]

There are still some clear problems with how it is separating. It should be much clearer on MNIST

[alyst]

Yes, I agree. I hadn't tried to reproduce it myself yet. I'm just trying to localize the issue a bit and understand whether it was like that for you back in April, or it's a new regression after the bugfix (there were no recent changes to TSNE, but BLAS/Julia version updates theoretically may affect the results).

[ChrisRackauckas]

It was like that back in April. I forgot to file that.

[alyst]

Ok, thanks! I plan to upgrade it to 0.7 when I have time in a next few weeks and then take a look at the issue. Unless somebody beats me to it.