ThummeTo / FMIFlux.jl

FMIFlux.jl is a free-to-use software library for the Julia programming language, which offers the ability to place FMUs (fmi-standard.org) everywhere inside of your ML topologies and still keep the resulting model trainable with a standard (or custom) FluxML training process.
MIT License
57 stars 15 forks source link

Avoid excessive tuple sizes #63

Closed maleadt closed 1 year ago

maleadt commented 2 years ago

While running this package's tests on PkgEval (testing the latest Julia master build), we've been seeing several segmentation faults during a type intersection of:

using LinearAlgebra, SparseArrays
x = Tuple{typeof(Base.hcat), Vararg{Union{Number, Array{T, 1} where T, Array{T, 2} where T, LinearAlgebra.AbstractTriangular{T, A} where A<:(Array{T, 2} where T) where T, LinearAlgebra.AbstractTriangular{T, A} where A<:Union{LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv} where T, LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(Array{T, 1} where T) where var"#s968", LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Hermitian{T, A} where A<:(Array{T, 2} where T) where T, LinearAlgebra.Hermitian{T, A} where A<:Union{LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Symmetric{T, A} where A<:(Array{T, 2} where T) where T, LinearAlgebra.Symmetric{T, A} where A<:Union{LinearAlgebra.Adjoint{var"#s968", var"#s967"} where var"#s967"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s968", LinearAlgebra.Bidiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Diagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.SymTridiagonal{T, V} where V<:AbstractArray{T, 1} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv} where T, LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(Array{T, 1} where T) where var"#s966", LinearAlgebra.Transpose{var"#s966", var"#s965"} where var"#s965"<:(SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv) where var"#s966", LinearAlgebra.Tridiagonal{T, V} where V<:AbstractArray{T, 1} where T, SparseArrays.AbstractSparseMatrixCSC{Tv, Ti} where Ti<:Integer where Tv, SparseArrays.SparseVector{Tv, Ti} where Ti<:Integer where Tv}, 354}}
y = Tuple{typeof(Base.hcat), Vararg{SparseArrays.SparseVector{Tv, Ti}}} where Ti<:Integer where Tv
typeintersect(x, y)

As noted in https://github.com/JuliaLang/julia/issues/47609, Julia generally isn't expected to handle types of that size.

ThummeTo commented 1 year ago

Dear @maleadt , thanks for noticing, we are on this now.

maleadt commented 1 year ago

Thanks!

ThummeTo commented 1 year ago

@JoKircher can you have an eye on this, too?

JoKircher commented 1 year ago

Will do, @ThummeTo

maleadt commented 1 year ago

FYI, here's a recent log of a crash caused by this: https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2022-11/19/FMIFlux.primary.log

ThummeTo commented 1 year ago

will be fixed in release v0.10.0

ThummeTo commented 1 year ago

@stoljarjo will further deploy a PkgEval-Action for the Github CI-Pipeline

JoKircher commented 1 year ago

Was this issue closed with https://github.com/ThummeTo/FMIFlux.jl/pull/79?

ThummeTo commented 1 year ago

almost, finally with v0.10.1 (pending) ;)

ThummeTo commented 1 year ago

closed with v0.10.1 #86