search

JuliaIO / JLD.jl

Saving and loading julia variables while preserving native types
MIT License
278 stars 55 forks source link

Save and read a sharedarray in a jld file #291

Closed PaulisBeetle closed 3 years ago

PaulisBeetle commented 3 years ago

When I save a two-dimension sharedarray in a jld fie and read the array from it again, the array becomes 

JLD.var"##SharedArrays.SharedArray{Base.Complex{Core.Float64},2}#7555"(JLD.var"##Distributed.RRID#7556"(1, 15227), (150, 180), [2, 3, 4, 5, 6, 7, 8, 9, 10, 11  …  142, 143, 144, 145, 146, 147, 148, 149, 150, 151], Any[JLD.var"##Distributed.Future#7565"(2, 1, 14927, nothing), JLD.var"##Distributed.Future#7565"(3, 1, 14928, nothing), JLD.var"##Distributed.Future#7565"(4, 1, 14929, nothing), JLD.var"##Distributed.Future#7565"(5, 1, 14930, nothing), JLD.var"##Distributed.Future#7565"(6, 1, 14931, nothing), JLD.var"##Distributed.Future#7565"(7, 1, 14932, nothing), JLD.var"##Distributed.Future#7565"(8, 1, 14933, nothing), JLD.var"##Distributed.Future#7565"(9, 1, 14934, nothing), JLD.var"##Distributed.Future#7565"(10, 1, 14935, nothing), JLD.var"##Distributed.Future#7565"(11, 1, 14936, nothing)  …  JLD.var"##Distributed.Future#7565"(142, 1, 15067, nothing), JLD.var"##Distributed.Future#7565"(143, 1, 15068, nothing), JLD.var"##Distributed.Future#7565"(144, 1, 15069, nothing), JLD.var"##Distributed.Future#7565"(145, 1, 15070, nothing), JLD.var"##Distributed.Future#7565"(146, 1, 15071, nothing), JLD.var"##Distributed.Future#7565"(147, 1, 15072, nothing), JLD.var"##Distributed.Future#7565"(148, 1, 15073, nothing), JLD.var"##Distributed.Future#7565"(149, 1, 15074, nothing), JLD.var"##Distributed.Future#7565"(150, 1, 
15075, nothing), JLD.var"##Distributed.Future#7565"(151, 1, 15076, nothing)], "/jl033629Bnkm00ml5mgvH7Qs7QH3", 
Complex{Float64}[-0.9485755962257433 + 0.31059367768546775im 0.05791363243049301 - 0.019120228740449077im … -5.638354778505679e-9 + 6.9269681943320935e-9im 1.5253859249286099e-7 - 1.8701269903407005e-7im; -0.792580904167772 + 0.6064375399264861im 0.05050079219900815 - 0.03846804706891346im … 2.1210511685768344e-9 + 2.4116132657932393e-8im -6.009035958181107e-8 - 6.74346068332579e-7im; … ; 0.6760417081343784 + 0.7343621709754525im -0.0416179943377734 - 0.04394333815793317im … 3.8770192410508395e-10 - 1.3360173080905237e-10im -1.0716812585119152e-8 + 3.51530512910819e-9im; -0.013420660597792667 - 0.011415589481678506im -0.06470011819598959 + 0.00813170732329344im … -0.022956213885446785 + 0.01657021979925804im -0.002796009686402473 + 0.01048169883248398im], 0, Complex{Float64}[])

How should I convert it into the origin data?

jmert commented 3 years ago

You'll need to unshare the data (collect into a single process) before saving; everything that was saved in that file is mostly metadata tracking where to find the remote data references. You can see the fields of the struct SharedArray in the output you've copied.

PaulisBeetle commented 3 years ago

Thanks! I have solved the problem.