JuliaAstro / FITSIO.jl

Flexible Image Transport System (FITS) file support for Julia
http://juliaastro.org/FITSIO.jl/
MIT License
55 stars 29 forks source link

Is `FITSIO` compatible with `SharedArrays` ? #114

Closed fincardona closed 3 years ago

fincardona commented 5 years ago

Hello,

I have a parallel code that exploits the module SharedArrays but when I try to write a SharedArray into a FITS file, I get the following error message:

ERROR: LoadError: MethodError: no method matching write(::FITS, ::SharedArray{Float64,1})
Closest candidates are:
  write(::FITS, !Matched::Array{String,1}, !Matched::Array{T,1} where T; units, header, hdutype, name, ver, varcols) at /home/users/federico.incardona/.julia/packages/FITSIO/BavHZ/src/table.jl:361
  write(::FITS, !Matched::Array{T,N} where N; header, name, ver) where T at /home/users/federico.incardona/.julia/packages/FITSIO/BavHZ/src/image.jl:166
  write(::FITS, !Matched::Dict{String,V} where V; units, header, hdutype, name, ver, varcols) at /home/users/federico.incardona/.julia/packages/FITSIO/BavHZ/src/table.jl:409
  ...
Stacktrace:
 [1] top-level scope at none:0
 [2] include at ./boot.jl:326 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1038
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] exec_options(::Base.JLOptions) at ./client.jl:267
 [6] _start() at ./client.jl:436
in expression starting at /gpfs/home/users/federico.incardona/julia_parallel_prova.Jl:31

So, is it possible to add a method to make FITSIO compatible with SharedArrays ?

giordano commented 5 years ago

This is interesting. We could change everywhere the type from {Vector,Array} to Abstract{Vector,Array}, but I don't know if the C library will like something different from the plain Array type.

kbarbary commented 5 years ago

Can you use sdata() on the shared array to get the underlying Array?

kbarbary commented 5 years ago

The C library requires a dense (or maybe strided) in-memory array. There are many AbstractArrays that do not fit those requirements, so I think AbstractArray would be too wide a signature.

jishnub commented 3 years ago

Note that this is not an issue anymore as FITSIO now accepts StridedArrays.

julia> s = SharedArray{Float64,2}((2,2), pids=workers());

julia> s .= reshape(eachindex(s), size(s))
2×2 SharedArray{Float64,2}:
 1.0  3.0
 2.0  4.0

julia> FITSIO.fitswrite("abc.fits", s) # works

julia> FITSIO.fitsread("abc.fits")
2×2 Array{Float64,2}:
 1.0  3.0
 2.0  4.0
giordano commented 3 years ago

That's great, thanks for checking (and your work on the strided arrays)! I'm going to close the ticket