inverse_transform of a PCA

[leonardtschora]

Hi everyone,

I noticed that the PCA imported from MultivariateStats does not have an inverse_transform method : info("PCA").implemented_methods yields

4-element Array{Symbol,1}: :clean! :fit :fitted_params :transform

This is causing the following code to produce an error:

a = rand(100)
M = DataFrame(a1 = a, a2 = 2a, a3 = 3a)

pca = PCA()
mach_pca = machine(pca, M)

fit!(mach_pca)
Mt = transform(mach_pca, M)
inverse_transform(mach_pca, Mt)

ERROR: MethodError: no method matching inversetransform(::MLJModels.MultivariateStats.PCA, ::MultivariateStats.PCA{Float64}, ::DataFrame)

Which is a bit annoying while wanting to integrate a PCA in pipelines or composite models (calling inverse_transform on the pipeline will fail because of the PCA step).

To my understanding, there is a reconstruct method provided by MultivariateStats, which allows to map the data back in its original space. Would it be sufficient du wrap it under a inverse_transform(model::PCA, fitresult, X) method?

Thanks for your help!

[leonardtschora]

After taking a dive into MLJModels.jl/src/MultivariateStats.jl I made this quick solution, which is working for my use case but I'm really unsure if this can be extendend to other cases.

function MMI.inverse_transform(::PCA, fr::PCAFitResultType, X)
    Xarray = MMI.matrix(X)
    Xnew   = permutedims(MS.reconstruct(fr, permutedims(Xarray)))
    return MMI.table(Xnew, prototype=X)
end

I just copied the already there transform method for PCA and replaced the transform by a call to reconstruct.

I attached my work file to this post (it includes basic tests etc...) : PCA.txt.

[leonardtschora]

Okay so it seems that I mixed up two problems at once.

The PCA models have no inverse_transform method and the propose one works fine for basic cases (see the file attached to the poste above).

But it seems that its also impossible to inverse_transform a pipeline with only Unsupervised transformations, even though the transformworks on a pipeline:

M = source((a=rand(100), b=rand(100)))
pipe = @pipeline Standardizer()
mach_pipe = machine(pipe, M)
xt = transform(mach_pipe, M)
xout = inverse_transform(mach_pipe, xt)
fit!(xout)

xt()
xout()

ERROR: type NamedTuple has no field inverse_transform Stacktrace: [1] getproperty(::NamedTuple{(:transform,),Tuple{Node{Machine{Standardizer}}}}, ::Symbol) at .\Base.jl:33 [2] inverse_transform(::Pipeline262, ::NamedTuple{(:transform,),Tuple{Node{Machine{Standardizer}}}}, ::NamedTuple{(:a, :b),Tuple{Array{Float64,1},Array{Float64,1}}}) at C:\Users\Leonard.julia\packages\MLJBase\2yoMe\src\operations.jl:103 [3] inverse_transform(::Machine{Pipeline262}, ::NamedTuple{(:a, :b),Tuple{Array{Float64,1},Array{Float64,1}}}) at C:\Users\Leonard.julia\packages\MLJBase\2yoMe\src\operations.jl:75 [4] (::Node{Machine{Pipeline262}})(; rows::Function) at C:\Users\Leonard.julia\packages\MLJBase\2yoMe\src\composition\learning_networks\nodes.jl:107 [5] (::Node{Machine{Pipeline262}})() at C:\Users\Leonard.julia\packages\MLJBase\2yoMe\src\composition\learning_networks\nodes.jl:107

[7] eval(::Module, ::Any) at .\boot.jl:331 [8] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86 [9] run_backend(::REPL.REPLBackend) at C:\Users\Leonard.julia\packages\Revise\BqeJF\src\Revise.jl:1184

Although it is still possible to connect everything by hand and get what I want, I think adding the possibility to inverse_transform a pipeline would be great.

Thanks for your support.

[ablaom]

Regarding implementation of inverse_transform for PCA

Opened an issue here: https://github.com/alan-turing-institute/MLJModels.jl/issues/291

Regarding inverse transform in a pipeline

Yes you cannot call inverse_transform on a @pipeline model, even if all the elements are transformers supporting an inverse_transform method.

However you can roll your own composite transformers that support inverse_transform (in addition to transform) using a learning network. Here's an example of a composite transformer that standardises and then scales (doubles):

using MLJ

## HELPERS

function double(table)
    names = schema(table).names
    A = MLJ.matrix(table)
    return MLJ.table(2*A, names=names)
end

function halve(table)
    names = schema(table).names
    A = MLJ.matrix(table)
    return MLJ.table(0.5*A, names=names)
end

## BUILD LEARNING NETWORK

X = source()  # wrap data here to "test as you build"

# composite transform:

stand = Standardizer()
stand_mach = machine(stand, X)
X1 = transform(stand_mach, X)

X2 = @node double(X1)

# composite inverse transform:

Z1 = @node halve(X)
Z2 = inverse_transform(stand_mach, Z1)

## EXPORT AS NEW MODEL TYPE

# define learning network machine:

mach = machine(Unsupervised(), X;
               transform=X2,
               inverse_transform=Z2) 

# define the new type:

@from_network mach begin
    mutable struct MyComposite
        standardizer=stand
    end
end

# use new model:

X = MLJ.table(rand(10, 3))

my_composite = MyComposite()
mach = machine(my_composite, X) |> fit!

Xnew = MLJ.table(rand(2, 3))
Z = transform(mach, Xnew)
Xnew2 = inverse_transform(mach, Z)
@assert Xnew2.x1 ≈ Xnew.x1

Closing in favour of https://github.com/alan-turing-institute/MLJBase.jl/issues/384 and https://github.com/alan-turing-institute/MLJModels.jl/issues/291

[leonardtschora]

Thanks a lot for your support! Just for the record : I was able to male my way through the problem using

function MMI.inverse_transform(::PCA, fr::PCAFitResultType, X)   
    permutedims(MS.reconstruct(fr, permutedims(X)))
end

(note that I skip the tabularization because here my data comes as a matrix and I need a matrix out). The problem using @nodes is that reconstruct expects a fitresult object, thus you can't create a nodethis way:

# Generate some data
X = rand(10, 10)
Xs = source(X)

# Create a PCA 
pca = PCA()
mach_pca = machine(pca, Xs)
Xr = transform(mach_pca, Xs)

# Format data to reconstruct it
X1 = matrix(Xr)
X2 = @node permutedims(X1)

# Reconstruct the data
Xb = @node MultivariateStats.reconstruct(mach_pca.fitresult, X2) ###
Xout = @node permutedims(Xb)
fit!(Xout); Xout()

Line ###will raise an error:

ERROR: UndefRefError: access to undefined reference

The decomposition:

# Make the retrieving of the fitresult a node
fr = @node getproperty(mach_pca, :fitresult)
Xb = @node MultivariateStats.reconstruct(fr, X2) 
Xout = @node permutedims(Xb)
fit!(Xout); Xout()

Is however working.

My solution here would be:

# Use X1 which is the reduced data matrixed
xout = inverse_transform(mach_pca, X1)
fit!(xout)
xout()

[OkonSamuel]

note that I skip the tabularization because here my data comes as a matrix and I need a matrix out

just to point out that you can wrap a matrix in a MatrixTable and request for the same matrix later on