BUG: ParticlePhasePlot does not redraw when _switch_ds is called

[mtryan83]

Bug report

Bug summary

The cookbook recipe (Time Series Movie) for animating plots suggests using plot._switch_ds(new_ds) to update the plot as part of the animation function. This does not work for ParticlePhasePlots (and likely PhasePlots in general). Relevant to #4288.

Code for reproduction

# This is copied directly from the cookbook with the modifications to create
# a semi-comparable ParticlePhasePlot animation
from matplotlib import rc_context
from matplotlib.animation import FuncAnimation

import yt

ts = yt.load("GasSloshingLowRes/sloshing_low_res_hdf5_plt_cnt_*")

#plot = yt.SlicePlot(ts[20], "z", ("gas", "density"),window_size=2)
plot = yt.ParticlePhasePlot(ts[0], ("gas","x"), ("gas", "y"), ("gas","density"), figure_size=2, x_bins=100, y_bins=100)
plot.set_unit(("gas","x"),"Mpc") # To match the SlicePlot units
plot.set_unit(("gas","y"),"Mpc")

plot.set_zlim(("gas", "density"), 8e-29, 3e-26)

fig = plot.plots[("gas", "density")].figure

# animate must accept an integer frame number. We use the frame number
# to identify which dataset in the time series we want to load
def animate(i):
    ds = ts[i]
    plot._switch_ds(ds)

animation = FuncAnimation(fig, animate, frames=len(ts))

# Override matplotlib's defaults to get a nicer looking font
with rc_context({"mathtext.fontset": "stix"}):
    animation.save("animation.mp4")

Actual outcome

https://user-images.githubusercontent.com/1833991/211651361-c05f464a-f339-4597-9880-576b085c3724.mp4

Note that the figure is updating, you can add a timestamp for example that will update correctly in each frame, but the underlying plot is not being replotted or something.

Expected outcome

https://user-images.githubusercontent.com/1833991/211652283-2db846ee-76ec-4e91-a6cf-4b8c4ea3c8bd.mp4

This was generated by generating each plot individually, saving them to separate files, and then using imageio to combine them into an mp4:

import os
import tempfile
import imageio
import yt

ts = yt.load("GasSloshingLowRes/sloshing_low_res_hdf5_plt_cnt_*")

with tempfile.TemporaryDirectory() as tempdirname:
    filelist = []
    # generate temporary figures
    for idx,ds in enumerate(ts):
        z_field = ("gas","density")
        p = yt.ParticlePhasePlot(ds, ("gas","x"), ("gas","y"), z_field,figure_size=2, x_bins=100, y_bins=100) 
        p.set_unit(("gas","x"),"Mpc")
        p.set_unit(("gas","y"),"Mpc")
        p.set_zlim(("gas", "density"), 8e-29, 3e-26)
        filename = os.path.join(tempdirname,f"ds{idx}.png")
        filelist.append(filename)
        p.save(filename)
    # build movie from files
    with imageio.get_writer('./animationFromSavedFiles.mp4', mode='I') as writer:
        for filename in filelist:
            image = imageio.imread(filename)
            writer.append_data(image)

Version Information

• Operating System: linux
• Python Version: 3.10.6
• yt version: main repo branch, 4.1.x
• Other Libraries (if applicable): NA

yt installed from source

[mtryan83]

@neutrinoceros suggested on slack that this was related to PhasePlot not having the export_to_mpl_figure method found in PWViewerMPL and copy/pasting the method fixes the issue (as a quick hack, not a final solution). I haven't had luck with that solution, though I might not have put it in the correct spot.

[neutrinoceros]

Thanks for reporting. So indeed my suggestion has no effect here because I was looking at a different animation example that relies on export_to_mpl_figure, which is currently not inherited by the PhasePlot class, but the example you're actually using doesn't do that. Sorry about the confusion !

So I went back to the example straight from the docs (using SlicePlot, and retriving the figure as fig = plot.plots[("gas", "density")].figure) and I can confirm that it works, which is a start.

Is the embedded mp4 supposed to be playable directly on GitHub ? I can seem to get it working in my browser.

[mtryan83]

Sorry about the embedded file, I accidentally deleted a new line after the file when I added the note. Should be fixed now.

I had looked at a different animation example as well (hopefully similar to what you were using) that relied on export_to_mpl_figure as well. Here's the test code I used:

import yt
from yt.testing import fake_amr_ds
from matplotlib.animation import FuncAnimation

yt.set_log_level("error")

# Modified from @neutrinoceros' cookbook example
# mock a time series
# *Note the seed parameter!*
ts = [fake_amr_ds(seed=s*100,fields=[("gas", "field1"),("gas","field2")], units=["code_length"]*2) for s in range(3)]
for i, ds in enumerate(ts):
    ds.current_time = ds.quan(i, "code_time")

z_field = ("gas","field1")
# SlicePlots (for example) work correctly
#p = yt.SlicePlot(ts[0],"z", z_field, window_size=2)
#p.annotate_timestamp() 

# Need to specify z field since ("all","particle_ones") does not exist
p = yt.ParticlePhasePlot(ts[0], ("gas","field1"), ("gas","field2"), z_field, figure_size=2) 
# PhasePlots don't have annotate_timestamp(), here's a quick & dirty version
time = ts[0].current_time.to("s")
timetxt = p.annotate_text(1e-5,1e-5,f"T={time}")
p.set_log("all",False)
p.set_zlim(("gas","field1"),0.1,1)

# Neither one of these work
#fig = p.plots[z_field].figure
fig = p.export_to_mpl_figure((1, 1))

def animate(i):
    print(f"{i=}", end="\r")
    ds = ts[i]
    #p.annotate_timestamp()
    time = ds.current_time.to("s")
    timetxt._plot_text[z_field]=f"T={time:0.3g}"
    p._switch_ds(ds)

animation = FuncAnimation(fig, animate, frames=len(ts), interval=500)
animation.save("./testFA.gif")
animation

This required a change to the fake_amr_ds method in lines 311 and 326:

# line 311
fields=None, units=None, geometry="cartesian", particles=0, length_unit=None, seed=0x4D3D3D3,
...
# line 326
prng = RandomState(seed)

This example still seems to show the same, non-updating behavior: Actual:

Expected (using the save each plot as file and recombine approach):

[neutrinoceros]

Thanks a bunch, I can read the new videos without problem. For some reason my favourite browser still doesn't want to play the ones in your OP, but I was able to watch them using another one.

It's getting late in my time zone, I'll do my best to have a real look at it tomorrow. In the mean times here are some pointers off the top of my head:

• Possibly a good way to start tracking this bug down would be to look for places where figures are being created in the PhasePlot/ParticlePhasePlot classes, and compare that with SlicePlot (possibly it's actually hard to track through class hierarchy).
• I wouldn't be surprised that this is caused by a missing or incorrect decorator from the @invalidate_data/@invalidate_plot/@invalidate_figure family (and there validate counterparts).
• another angle would be to look for major differences in those classes' respective render methods

[neutrinoceros]

Been hacking at it for two hours tonight, here's what I think is happening: the data which is held by a PhasePlot (or ParticlePhasePlot, the parts I care about here are not overridden in the child class) lives in self.profile, but this attribute is not updated when _switch_ds is called. I find that overriding the _switch_ds method for PhasePlot resolves the "nothing is happening" situation and I can get the plot to update:

https://user-images.githubusercontent.com/14075922/211934711-45570b84-fe95-4a0a-b4b2-1494d5f70f8c.mp4

I'll open a PR with my patch

[neutrinoceros]

@mtryan83 my patch is in #4295, feel free to try it out and/or give some feedback.