MPI Volume Rendering - Single File Write

[cbyrohl]

Bug report

When rendering a scene with yt using MPI and wanting to write the image to disk on rank==0 only, the inspired recipe from the documentation (https://yt-project.org/doc/analyzing/parallel_computation.html#creating-parallel-and-serial-sections-in-a-script) leads to a freeze.

While the example given in the documentation uses a ProjectionPlot as example, I think the documentation page's idea here is to provide a somewhat generic recipe that should also hold for Scenes.

Maybe something can be done about Scene.save() which states

If an image has not yet been rendered for the current scene object, it forces one and writes it out.

but the render is called irrespective of a current scene object having been rendered before thus leading to a freeze, see below.

Code for reproduction

I use a custom scene "sc", which I then call render() on, which is executed by the different ranks. Finally, I only want the root rank to write the rendered image. I attached a minimal snippet below. Note that this snippet uses yt.volume_render() where every rank already writes an image to disk, which of course would make the final write redundant. In reality, often a scene is manually created so that this automatic write per rank does/should not occur.

import yt

yt.enable_parallelism()
# Load the dataset.
ds = yt.load("Sedov_3d/sedov_hdf5_chk_0001")

# Create a volume rendering, which will determine data bounds, use the first
# acceptable field in the field_list, and set up a default transfer function.

# This will save a file named 'data0043_Render_density.png' to disk.
im, sc = yt.volume_render(ds, field=('gas', 'density'))

if yt.is_root():
    sc.save("MPI.png")

Execute this snippet with at least two MPI ranks.

Actual outcome

The execution freezes as a new sc.render() is called in sc.save() and only the root process entered the if-clause.

Expected outcome

Write the image through rank==0 without freeze.

Version Information

• Operating System: SUSE Linux Enterprise Server 15 SP1
• Python Version: 3.7
• yt version: github dev

[welcome[bot]]

Hi, and welcome to yt! Thanks for opening your first issue. We have an issue template that helps us to gather relevant information to help diagnosing and fixing the issue.

[chrishavlin]

This is somewhat mitigated now that you can do:

if yt.is_root():
    sc.save("MPI.png", render=False)

and it won't hang. but ideally running sc.save("MPI.png") should not hang indefinitely, so I think this issue should stay open for now (it'd also be good to add a note on volume rendering in parallel that mentions this)