Playing Markdown to include dynamic content such as SQL tables

[shah]

@cdaringe I was curious if markdown_player might support this feature and I'm missing it:

• I'd like to have a Markdown "template" file called my-markdown.md
• my-markdown.md would have fenced code blocks of SQL inside comments and named blocks for results of SQL captured by running psql (PostgreSQL) command from bash
• I'd like to mark that my output from the SQL command is itself Markdown and should be "included" here from the first SQL. If we can "capture" the output as arbitrary Markdown and include it that would be great since I can take SQL output and turn it into a table or anything else.
• I'd like the content saved back into my-markdown.md, preserving the SQL in comments but replacing the text that was executed by SQL.
  • This way, the latest file has the template and the merged output, ready for running the next time. Now I can teach my users to incorporate SQL directly into my-markdown.md and be able to "play" the markdown with the same SQL but emitting different output. Then, we just stored the file right back into Git/file store and we have only one file that acts as the template + the output whenever we need.

For example: my-markdown.md

# My markdown heading

Introductory text for my document.

## A second heading

Some more text

## The SQL output I care about heading

<!--
'''sql {cmd: psql, args: ["--c", $ARG], capture: "query1"}
select * from table
'''
-->

<!-- { capturedOutput: 'query1', position: 'start' } -->
NOTE to Christopher: somehow I'd like to mark that my output from the above command is itself Markdown 
and should be "included" here from the first SQL. If we can "capture" the output as arbitrary Markdown and 
include it that would be great since I can take SQL output and turn it into a table or anything else. 
When "played" markdown_player would replace everything between start/end of captured output for each 
run but leave the rest of the markdown content as-is.
<!-- { capturedOutput: 'query1', position: 'end' } -->

Some more markdown text.

<!--
'''sql {cmd: psql, args: ["--c", $ARG], capture: "query2"}
select * from another_table
'''
-->

<!-- { capturedOutput: 'query2', position: 'start' } -->
NOTE to Christopher: somehow I'd like to mark that my output from the above command is itself Markdown
and should be "included" here from the second SQL.
<!-- { capturedOutput: 'query2', position: 'end' } -->

Some more markdown text.

Thoughts? :-)

[cdaringe]

greetings!

• the --appendOutput flag will update output blocks, which you've likely discovered, and is active in this project's readme
• as it pertains to providing a full output block as markdown, i'd be open to that
  • the relevant code is here: https://github.com/cdaringe/markdown_player/blob/0b132c5e815ae59030b74800a0620b8eb4b4ddbc/src/mod.ts#L62-L76
  • the appending or updating of output content works naively by checking if an output fence follows an input/executed code fence, and if the output fence has meta on it of isExecutionOutput

so, were such a feature to be added:

• the output markdown would need to probably self-provide meta with those values, or, this lib would need to mutate your markdown's meta with that content
• some mechanism would need be added to communicate the intent of self-hosted/defined markdown. i'd be open to ideas and patches on that

seems like a useful feature. minimally, defining the language type for the output markdown seems very valuable, and that is currently lacking. perhaps an output: ... field on the input markdown's codefence would be where you/we could experiment w/ such an api?

[cdaringe]

fwiw, i have intent to use this against postgres as well :) i wrote a very mediocre article some time ago that i still present periodically to peers (https://cdaringe.com/protect-bacon-with-rls) where i advocate for DB reflecting APIs with security all at the data layer. i had this lib in mind recently when brushing up on that content. so, i want this feature too :)

[shah]

Got it @cdaringe I'll take a peek to see if I can propose some changes via PRs. Meanwhile, you and I agree on the database being a proper computing platform. Check out my code on that subject here: https://github.com/netspective-studios/pgdcp :-)