search

JuliaIO / JLD2.jl

HDF5-compatible file format in pure Julia
Other
537 stars 84 forks source link

Out of disk space throws generic Bus Error #507

Open ejmeitz opened 8 months ago

ejmeitz commented 8 months ago

Hello,

I'm not totally sure how to reproduce this but I have some code that looks like the snippet below and it crashes in the call to jldsave with a Bus error. Is there a reason this happens? I don't see why I can't write unique files in a multithreaded manner. Thanks.

@sync for i in 1:4
     @async begin
          ....
            for j in 1:100
                 jldsave('unique_path')
            end
           ......
     end
end
ejmeitz commented 8 months ago

Ok I realized I made a post like this awhile back and I am out of disk space. Is there some way to have JLD2 relay this information in the crash message?

JonasIsensee commented 7 months ago

Hi @ejmeitz,

I don't know but also don't think so. This error is thrown (i think) when JLD2 tries to increase the Mmap file size. (ftruncate ?) I'm not sure if there one can ask the OS whether an increase will be granted. (With multiple tasks that's more difficult to guarantee working)

yha commented 4 months ago

Maybe related to julia issue with propagating full disk errors? https://github.com/JuliaLang/julia/issues/44535