search

DCsunset / pandoc-include

A pandoc filter to allow file and header inclusion
MIT License
63 stars 14 forks source link

fix: make temporary filename unique. #11

Closed W-Floyd closed 1 year ago

W-Floyd commented 3 years ago

This makes it possible to run pandoc with GNU parallel.

DCsunset commented 3 years ago

Hello, thanks for your contributions. This PR is similar to #10.

The temp file now is used to share config between multiple executions of pandoc_include.py, i.e. the subsequent executions of pandoc_include.py need to share the same temp file of its parent. I think a unique temp file will break the original functionaily.

W-Floyd commented 3 years ago

I guess my understanding of the program is incorrect.

What I am trying to achieve is making it possible to run pandoc in parallel, while still using this filter. Currently, if I do so, it overwrites the same file for different runs, which kills it.

W-Floyd commented 3 years ago

Running the test produces the same outputs with the change I added. Why might this be?

DCsunset commented 3 years ago

The test doesn't cover this feature, so the result might be the same in this case.

This feature mainly provides convenience for pandoc-options. For example, if you want to use some other filters when including other files, it would be cumbersome to specify the pandoc-options in every included file. So by default, the included files will inherit their parents' pandoc-options.

As for parallelism, I don't know whether there's a good workaround. Do you have any idea?

W-Floyd commented 3 years ago

Ah I see, that makes sense.

I'll admit to not being familiar at all with pandoc filters, but I take it that it's not possible to pass a filename on down the chain to subsequent runs of the filter? Ideally, could one set a filename on the first run, and propagate that?

DCsunset commented 3 years ago

I haven't found a feasible way to run a pandoc filter with some argument, especially inside a filter. A possible solution is not to use a temp file by default and only use it when changing the pandoc-options. In most cases, the pandoc-options won't be changed, so you could run it in parallel.

However, when you change the pandoc-options, you cannot run it in parallel. What do you think of this solution?

W-Floyd commented 3 years ago

That sounds like a reasonable compromise. Too bad there's no way to pass options, but I understand. Thanks for your work!

From: DCsunset notifications@github.com Sent: Wednesday, November 25, 2020 2:35:39 AM To: DCsunset/pandoc-include pandoc-include@noreply.github.com Cc: William Floyd william_png2000@yahoo.com.au; Author author@noreply.github.com Subject: Re: [DCsunset/pandoc-include] fix: make temporary filename unique. (#11)

I haven't found a feasible way to run a pandoc filter with some argument, especially inside a filter. A possible solution is not to use a temp file by default and only use it when changing the pandoc-options. In most cases, the pandoc-options won't be changed, so you could run it in parallel.

However, when you change the pandoc-options, you cannot run it in parallel. What do you think of this solution?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/DCsunset/pandoc-include/pull/11#issuecomment-733552920, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACZ5PFBXBMP7XWCCE2P7AKLSRS6VXANCNFSM4TT73YFA.

DCsunset commented 3 years ago

I have applied the above workaround in the master branch. Could you give the command to run it with GNU parallel so I could test it?