search

jgm / pandoc

Universal markup converter
https://pandoc.org
Other
33.14k stars 3.3k forks source link

Lua support for resolving files in the data directory #9854

Closed jeffvalk closed 3 weeks ago

jeffvalk commented 3 weeks ago

Add Lua support for resolving and reading files from the data directory. At minimum, this should support loading named templates. Ideally, this would enable API access to any type of file stored in the data directory.

From discussion https://github.com/jgm/pandoc/discussions/9853:

It would feel more "batteries included" in the API, and it would give filter/reader/writer authors an indication that they're working with Pandoc's architecture and not against it.

Thoughts:

  1. Add new function get(template_name) to module pandoc.template, which locates the named template in the "templates" directory, and reads and returns its text content, similarly to existing function pandoc.template.default.

  2. Add new function data_file(name) to module pandoc.utils (or elsewhere), which resolves a relative path in the data directory and returns an absolute file path if the file exists, or nil if does not. This would provide a standard mechanism to allow any data directory file to be read, added to the media bag, etc.

tarleb commented 3 weeks ago

I've opened #9855.

jeffvalk commented 3 weeks ago

Awesome.

Do you think item 2 above is worthwhile?

tarleb commented 3 weeks ago

I think a read_datafile function or similar would be a useful addition. What's keeping me from adding right away is that, for one, I'm not sure where to place it, as the utils module seems a bit too random for it. The other is that "manually" reading it (or using pandoc.mediabag.fetch) seems less bad for generic data files, and is probably a rarer operation, too.

There's also the generally conservative approach with respect to new Lua functions, as each new functions adds more code to maintain (and compile, and ship), so it's important to only add the "right" functions.

BTW, that's also the reason why I open PRs for these functions instead of just pushing a commit to master, because I want to make sure that "we get it right" with these API changes.

tarleb commented 3 weeks ago

So what I'm trying to say is that I wouldn't rule out data_file, but that the current available mechanisms are acceptable. Adding a function would just be for convenience.

The pandoc.template.get function on the other hand makes the mechanism behind the --template parameter usable in Lua. The parameter uses code that's not necessarily obvious, and is also potentially subject to change. Then the Lua function should change with it. That's why I favor adding that one for now.

jeffvalk commented 3 weeks ago

That all makes perfect sense. I had some of the same thoughts, which is why I asked.