JackMcKew / pandas_alive

Create stunning, animated visualisations with Pandas & Matplotlib as easy as calling `df.plot_animated()`
MIT License
582 stars 100 forks source link

Parallelise frame writing to speed up rendering #13

Closed owenlamont closed 4 years ago

owenlamont commented 4 years ago

Given rendering movies with hundreds to thousands of frames can be quite time expensive I was wondering about the possibility of farming the rendering out to multiple processes - maybe using the joblib package.

I know in principle this should work well - but I'm not sure how difficult it will be to integrate with Matplotlib's animation logic. You derive from the FuncAnimation class which looks like it assumes sequential updates and doesn't seem to lend itself to being parallelised. If we used the Writer classes or particularly the FileWriter classes those seem to have more obvious ways to parallelise but even then it might require some tweaks to Matplotlib itself to support rendering frames in parallel - and the refactoring might be too big for pandas_alive.

Would welcome your thoughts, this may be too far out of scope or maybe it should be an issue raised with Matplotlib itself - but pandas_alive case seemed more specialized where we know each frame corresponds to the row of a Dataframe - the general animation case might not have the state of all frames so well defined so I thought I'd run it by you first in the context of pandas_alive.

JackMcKew commented 4 years ago

Pandas_Alive intends to use other packages Writer classes, any modification to the behaviour of how they work would be best done in the package responsible.

Given how much development time would be needed for this feature, I'm not sure if the benefits outweigh the time required.

owenlamont commented 4 years ago

That's cool, I suspected it would be more trouble than it was worth here. You can close this issue.