Macros to List All Built-In and User-Defined Macros

[tajmone]

Issue #40 made think that if the \ syntax was to be dropped, I'd need to do some RegEx search-&-replace on all my projects folders to adapt to the new syntax. In SublimeText this is rather easy to achieve using file patterns (covering both markdown files and PP definitions files — I use "*.pp" for them).

The painful part would be to write the RegEx to capture all the built-in macros and the custom definitions.

This made me think that having a built in macro that can emit a list of all the built in macros, and another to emit a list of all the user defined macros, could be handy — not only for the above mentioned task, but also for maintaing documentation on PP or custom macros.

Something like this:

A !list-builtin-macros, that would emit:

def
define
[.. etc ...]
src

(covering both short and long version of each macro syntax)

And a !list-user-macros, that would emit:

MyMacro
AnotherMacro
[.. whatever ...]

The latter could be invoked after importing all the macros definitions files, and then emit the full list of user-defined macros (for example, in the project I'm currently working on, I have quite a lot of custom macros definitions, and they are invoked hundreds of times in the markdown sources).

Both lists could then be used to easily create the needed search and replace RegEx:

\\(def|define|...list of macros...|src)

... with a replace pattern:

!$1

... but I can envisage other uses also.

[CDSoft]

I was thinking about something similar to generate the documentation of builtin macros and a better command line documentation. I wanted to add a docstring to builtin macros instead of hard writing the documentation in pp.md.

For user macros I can track each definition and emit them in various formats (simple list, plain text with doc and markdown with doc). !def could take an optional parameter to document the macro and produce some documentation in the same format than builtin macros.

[tajmone]

!def could take an optional parameter to document the macro and produce some documentation in the same format than builtin macros.

... this is a cool idea. I currently put each macro description and usage instruction in a !comment block, these could be easily converted into the new system.

This is an example file of how I'm currently using comments to document macros:

• https://github.com/tajmone/pb-archives-test/blob/source/_butler_/macros/Highlight.pp

Usually, I place the description before the definition; and use tilda or backtick fences as parameter delimiters, so the boundries stand out visually.

The Fenced-Paramater Dilemma

I guess that adding an optional paramenter to !def will introduce the dilemma of which param comes first, the def or the description? Because only the last param can be delimited by tildas; and an optional param would have to either come last (and disallow tildas fencing on the definition) or force the user to pass empty brackets when unneed (!def(mymacro)()(macro definition) ) and disallow fencing for the description:

!def( MACRO NAME )( [OPTIONAL DOCS] )
~~~~~~~~~~~~~~~~~~
MACRO DEFINITION
~~~~~~~~~~~~~~~~~~

!def( MACRO NAME )( MACRO DEFINITION )
~~~~~~~~~~~~~~~~~~
OPTIONAL DOCS
~~~~~~~~~~~~~~~~~~

Two Separate Macros: !def and !defdoc

Tildas fencing might be necessary for both the macro definition and description, when white space preservation is required. For this reason, it might be better to have a separate macro that can add/modify the description/documentation to a user defined macro (even if not yet existing, so that in the definition file it could precede the actual definition) :

!defdoc(mymacro)
~~~~~~~~~~~~~~~~~~
the macro's documentation
~~~~~~~~~~~~~~~~~~

!def(mymacro)
~~~~~~~~~~~~~~~~~~
the macro's definition
~~~~~~~~~~~~~~~~~~

This approach of separate macro/docs definitions would also allow more freedom in creating conditional statement for defining/redifining different macro documentation texts according to the locale or format (eg: via !lang or !format).

For this same reason, it might also be a good idea to allow the user to overwrite the description text of built-in macros — eg: for easier maintainance of documentation in other locales, where he could translate all of the built-in macros' default texts in a single file.

---

SIDE NOTE: I currently keep the comment blocks outside the macro definition, because (if I understood correctly Issue #32) a comment inside a definition would end up stored in memory as part of the definition, and add some memory bloat:

The definition is the original macro definition stored in a simple string that is preprocessed each time the macro is called.

[CDSoft]

My goal is to generate short documentation (for the README or the command line). So a single macro with one line doc (most of the time) is enough:

  !def( MACRO NAME )( SHORT HELP TEXT )
  ~~~~~~~~~~~~~~~~~~
  MACRO DEFINITION
  ~~~~~~~~~~~~~~~~~~

  !def( MACRO NAME )
  ( LONGER SHORT HELP TEXT
    ON TWO LINES...
  )
  ~~~~~~~~~~~~~~~~~~
  MACRO DEFINITION
  ~~~~~~~~~~~~~~~~~~

SIDE NOTE: Memory is cheap. Haskell consumes a lot of memory. Adding comments in macros won't change anything.

[CDSoft]

The version 2.1 generates builtin and user macro help from the source code:

pp -help => only builtin macros
pp -import foo -help => builtin macros and user macros defined in foo

The last four macros are:

macros
    !macros lists the builtin macros.

usermacros
    !usermacros lists the user macros.

help
    !help prints built-in macro help.

userhelp
    !userhelp prints user macro help.

[robinrosenstock]

Is this issue resolved? If yes, it should be closed.

[tajmone]

Sorry, I had planned to test this feature more deeply but didn't find the time to.

So, one last question: if I defined the help message in a temp macro, using fenced delimiters, and then pass that macro as HELP TEXT, will I manage to overcome the above mentioned limitation, and achieve a long and verbatim help text for user-defined macros?

Example

!def(MY_HELP_TEXT)
~~~~~~~~~~~~~~~
Some long, verbatim help text.

   White space preserved!
~~~~~~~~~~~~~~~

!def( MY_MACRO )( !MY_HELP_TEXT )
~~~~~~~~~~~~~~~
MACRO DEFINITION
~~~~~~~~~~~~~~~

Will this work?

(couldn't find time to test this myself in these days, the recent update of pandoc to v2.x required updating lots of code and documents, and still haven't managed to get them all done)

[CDSoft]

This works but maybe not as you expect:

$ cat issue41.md 
!def(MY_HELP_TEXT)
~~~~~~~~~~~~~~~
Some long, verbatim help text.

   White space preserved!
~~~~~~~~~~~~~~~

!def( MY_MACRO )( !MY_HELP_TEXT )
~~~~~~~~~~~~~~~
MACRO DEFINITION
~~~~~~~~~~~~~~~

!userhelp

$ pp issue41.md

**`MY_MACRO`**
:   Some long, verbatim help text. White space preserved!

The help text is supposed to be a single paragraph.

[tajmone]

Mhhh... I thought that usually the help would contain a syntax/usage example (which often might require fenced delimiter examples), so I would have expected white space preservation.

But I see your point, the idea is to inject into documentation a brief resume of all the macros, in a nicely preformatted way. It makes sense.