JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.92k stars 5.49k forks source link

sparse is twice as slow with Int32 indices #12963

Closed bdqp closed 9 years ago

bdqp commented 9 years ago

Constructing a large sparse matrix, using Int32 indices is a lot slower. Is this a performance bug or expected behavior? I kind of thought it would be quicker due to more elements being available in cache memory.

julia> N=1000000; NNZ=20*N; julia> r = rand(int32(1:N),NNZ); julia> c = rand(int32(1:N),NNZ); julia> v = rand(Float32,NNZ); julia> tic(); A = sparse(r,c,v,N,N); toc() elapsed time: 11.829277923 seconds 11.829277923

julia> r = rand(int64(1:N),NNZ); julia> c = rand(int64(1:N),NNZ); julia> tic(); B = sparse(r,c,v,N,N); toc() elapsed time: 6.594016314 seconds 6.594016314

julia> whos() A 1000000x1000000 SparseMatrixCSC{Float32,Int32} B 1000000x1000000 SparseMatrixCSC{Float32,Int64}

KristofferC commented 9 years ago

There is apparently a type instability for sparse when it is called with Int32s.

julia>  @code_warntype  Base.SparseMatrix.sparse(r,c,v,N,N, Base.AddFun());
# .
# pdest::Any
#  .

Edit:

pdest starts of as an Int32 but gets promoted to an Int64 at https://github.com/JuliaLang/julia/blob/master/base/sparse/csparse.jl#L89

KristofferC commented 9 years ago

Just a fyi, sparse is significantly faster on 0.4 due to https://github.com/JuliaLang/julia/pull/10719 if you need to accumulate values.

bdqp commented 9 years ago

Wow you guys are on it! Thanks xxx ;)