search

LaurentRDC / pandoc-plot

Render and include figures in Pandoc documents using your plotting toolkit of choice
https://laurentrdc.github.io/pandoc-plot/
GNU General Public License v2.0
216 stars 8 forks source link

Concurrency issue when generating identical graphs #53

Closed ewft closed 1 year ago

ewft commented 1 year ago

Hi,

I found out that if a document contains multiple identical plots ( exactly same source code to generate the plot), pandoc-plot fail ( in a random way) over a race condition on the created files.

pandoc-plot: plots/pandocplot12097460476205103681.src.html: withFile: resource busy (file is locked).

I work around this by adding some random comment different in each plot code source.

I run across that issue when dealing with automated generated document (Test report) with identical plots ( basic pie chart showing the test results (PASS/FAIL).

I think this could be solve by adding a random part in the calculation of the plot ID. And probably conditioned by an option to not impact performance when not needed.

LaurentRDC commented 1 year ago

Hi Ben,

Thank you for your bug report.

This is because the content of each script is hashed, and the hash determines the filename. This means that two scripts with the same content will be exported to the same file, as you pointed out.

The reason for this was to prevent pandoc-plot from re-generating plots on large documents. So if I change a single plot script in a 500 pages book, only 1 plot will be re-generated. If pandoc-plot added a randomly-generated string to every plot, then I would lose the 'caching' feature.

If you have N identical plot scripts (all with an identical hash value), then pandoc-plot could generate only 1 figure. Then, in the final document, you would have N links to the same file. Would that solution work for you?

LaurentRDC commented 1 year ago

I've opted for the crude fix for now: add a lock to file writing operations. Most of the time is spent rendering figures anyways, so this lock won't have much effect on performance.

Once the tests pass, including a new one replicating the issue you reported, I'll make a new release.

LaurentRDC commented 1 year ago

Version 1.5.5 has just been released. It should fix your issue. Please give it a try and let me know how it goes.

LaurentRDC commented 1 year ago

I'll close this for now, as version 1.5.5 should fix this issue. If not, please don't hesitate to re-open the issue.