kleinhenz / SlurmClusterManager.jl

julia package for running code on slurm clusters
46 stars 5 forks source link

Output file is not populated in real time #4

Closed matthiasbe closed 3 years ago

matthiasbe commented 3 years ago

Consider the following julia script

#!/path/to/julia --project

using SlurmClusterManager
using Distributed

addprocs(SlurmManager())

i = 0
for i in 1:10
        sleep(1)
        fetch(@spawnat (workers()[i % length(workers()) + 1]) println(i))
end

When I run this on my local session, it prints every second.

When I run this on the cluster with the slurm manager, it print everything at the termination of the job. This makes it difficult to track the progress of the job.

matthiasbe commented 3 years ago

This does not come from the Cluster Manager. adding flush(stdout) for each printing solve the problem.

More info https://discourse.julialang.org/t/how-to-flush-stdout/10146