Add support for positive semidefinite matrices

[mschauer]

This is the sister issue of JuliaStats/Distributions.jl/issues/366 about normal distributions with degenerated covariance matrix. On implementation would be a famility of AbstractPSDMat's . The question is if this should also contain the concrete types now in the AbstractPDMat hierarchy. There are two natural choices of matrix factors, Cholesky with pivoting and LDL' decomposition, available form base respective from LAPACK.

[lindahua]

What about having AbstractPDMat <: AbstractPSDMat?

[mschauer]

Yes, that's nicest because it incorporates reality into specification.

[mschauer]

This is fixed for the general PDMat constructor which does not check positive definitiness:

julia> L = SMatrix{2,2}(1,2,0,0);rand(MvNormal(PDMats.PDMat(L*L', Base.LinAlg.Cholesky(L, :L))))
2-element Array{Float64,1}:
 1.35396
 2.70793

Is this now officially supported?

[iamed2]

~The Cholesky constructor does still check positive definiteness~ nvm, only true for Julia 0.6

[nickrobinson251]

Reviving this issue :)

Would you be open to a PR adding a PSDMat type?

We have some private code that defines a PSDMat <: AbstractPDMat (i believe written originally by @iamed2 and @rofinn)

struct PSDMat{T<:Real,S<:AbstractMatrix} <: AbstractPDMat{T}
    dim::Int
    mat::S
    chol::CholeskyPivoted{T, S}

    function PSDMat{T,S}(d::Int, m::AbstractMatrix{T}, c::CholeskyPivoted{T, S}) where {T, S}
        return new{T, S}(d, m, c)
    end
end

We could also consider the type hierarchy here. e.g.

1. just define this type as <: AbstractPDMat
2. as above, but also rename AbstractPDMat -> AbstractPSDMat
3. define a new AbstractPSDMat <: AbstractMatrix, and define a hierarchy like PSDMat <: AbstractPSDMat and AbstractPDMat <: AbstractPSDMat. This would then require PRs to loosen type constraints in e.g. Distributions to now be AbstractPSDMat.

[nickrobinson251]

there's now a PSDMat in https://github.com/invenia/PDMatsExtras.jl