Prepend -r & -d command line registers requests after macro files, permitting user override

[rnkn]

Sorry for my delay in responding to #111 and #112. Thanks for going ahead without waiting around.

I found the documentation on the original (non-GNU) ms macro file here: https://troff.org/using-ms.pdf

The list of registers is on p5, Registers.

But I had a thought... given the the -m flag is just prepending the a macro file to the input file, instead of totally rewriting the ms file (and the mm file, and the eqn file, etc.) to be more like GNU, would it make more sense to instead make troff convert the command line registers set with -ra=N and -da=N into roff requests and prepend these after the macro file?

I assume what's currently happening is the command line registers are being prepended before the macro file, hence the macro overriding them.

If prepending after the macro file, then troff -ms -rPS=12p -rVS=1.4v -dCF=% source.ms would send the following to troff:

.\"<contents of s.tmac>
.nr PS 12p
.nr VS 1.4v
.ds CF %
.\"<contents of file.ms>

This way the command line registers override the macro file instead of vice versa, and so will work with any macro file.

Thoughts?

[n-t-roff]

Have you tested, that it works?

[Alhadis]

This would make it impossible to configure a macro package's behaviour at startup. It also renders the -r switch useless, since users can already specify "post-startup behaviour" by loading multiple macro files, or by embedding an .so request that loads a shared configuration file.

I'm not sure I understand the problem that you're attempting to solve.

[rnkn]

(Thanks for the quick replies; somehow I was not subscribed to my own issue...)

Hmm I think I was unclear...

Currently, loading a macro file appends that macro file to the source file. The existing macro files define their own registers, e.g. ms uses CF for the centre footer string.

The problem is that, currently, setting a macro's registers on the command line gets ignored — or mostly ignored, except for the registers recently added to ms in #112.

This is because the macro file is appended to the head of the source file and thus the register definitions are read after any command line flags — the macro file is overriding the command line flags. This contradicts a user expectation that their commands should be honoured by the computer.

What I am suggesting is realigning the order in which registers are defined, that is, registers from command line flags are defined last such that they override any definitions already present in macro files. This aligns more with user expectation of commands issued to a computer.

The more general use-case is employing troff as intermediate in a command chain where the originating file is not in roff format, e.g. converting from Markdown to roff to Postscript. From reading the documentation it would appear the correct way to set the macro registers in this case is via -r and -d command line flags (but as we've found, this mostly fails).

The solution I'm actually using is to cat a configuration file onto the top of the intermediate roff and therefore between the source and the appended macro file:

< infile.md lowdown -Tms | cat config.ms - | troff -ms | dpost > outfile.ps

This works, but from what you're saying @Alhadis should I be able to treat this configuration file just like a macro file, and append it in order using -m?

[Alhadis]

[…] the macro file is overriding the command line flags. This contradicts a user expectation that their commands should be honoured by the computer.

I should point out that the classical macro packages—ms, mm, me, and man—are all ancient, and likely predate the introduction of an -r switch. Many of the original registers that were used to customise documents are still supported by modern versions of these packages, and it was only relatively recently that some versions (i.e., Groff's) were refactored to take advantage of registers set from the command-line.

In other words, had ms been written today, I guarantee its design would be more ergonomic (with important registers not being overwritten when the package is loaded). The same can be said of the other classical macro packages.

This works, but from what you're saying @Alhadis should I be able to treat this configuration file just like a macro file, and append it in order using -m?

You can in Groff. Heirloom, sadly, appears to restrict the user to only one macro package selected via -m (and further limits its filename and filesystem location). You should really be using soelim(1) and/or .so requests to link files together this way.

[rnkn]

I should point out that the classical macro packages—ms, mm, me, and man—are all ancient, and likely predate the introduction of an -r switch. Many of the original registers that were used to customise documents are still supported by modern versions of these packages, and it was only relatively recently that some versions (i.e., Groff's) were refactored to take advantage of registers set from the command-line.

In other words, had ms been written today, I guarantee its design would be more ergonomic (with important registers not being overwritten when the package is loaded). The same can be said of the other classical macro packages.

Yeah I felt kinda bad about pushing to alter the ms macro file in #111. I kinda feel like these are more "heirlooms" than the troff program itself.

You can in Groff. Heirloom, sadly, appears to restrict the user to only one macro package selected via -m (and further limits its filename and filesystem location). You should really be using soelim(1) and/or .so requests to link files together this way.

Ah okay. Does this do anything different to just cat? Both of these seem effectively the same:

$ < infile.md lowdown -Tms | cat config.ms - | troff -ms | dpost > out.ps
$ < infile.md lowdown -Tms | soelim config.ms - | troff -ms | dpost > out.ps

[Alhadis]

Ah okay. Does this do anything different to just cat? Both of these seem effectively the same:

soelim simply expands .so requests with the contents of the files they reference. It's really only necessary in certain situations. From soelim(1):

soelim was designed to handle situations where the target of a roff source request requires a preprocessor such as eqn(1), pic(1), refer(1), or tbl(1). […] That is, files sourced with .so are normally read only by troff (the actual formatter). soelim is not required for troff to source files.

Anyway, I've no idea what lowdown is, but if you're authoring your document using Markdown, you're gonna have a bad time. 😜

[rnkn]

soelim simply expands .so requests with the contents of the files they reference. It's really only necessary in certain situations. From soelim(1):

soelim was designed to handle situations where the target of a roff source request requires a preprocessor such as eqn(1), pic(1), refer(1), or tbl(1). […] That is, files sourced with .so are normally read only by troff (the actual formatter). soelim is not required for troff to source files.

Haha that man page is for a different version of soelim to the Heirloom doctools one! The Heirloom soelim man page is a bit more terse.

So, for purposes of discussion, soelim is a red herring here, because it doesn't affect the initial example of:

troff -ms -rPS=12p -rVS=1.4v -dCF=% source.ms

Anyway, I've no idea what lowdown is, but if you're authoring your document using Markdown, you're gonna have a bad time. 😜

You seem to really like red herrings! I quite like lowdown, but for our purposes it doesn't matter where the initial input is coming from, as long as it's from STDIN. So really the example should be thought of as something like this:

cat source.ms | troff -ms -rPS=12p -rPD=0 -dCF=% | dpost > out.ps

In this case -rPS will be honoured thanks to #112 but -rPD and -dCF will be ignored.

One solution would be to rewrite the macro files (as GNU did) but as you said, they are ancient, and the project is called "Heirloom".

What I propose is just to switch the order of the code the appends macro files vs command line registers.

While I confess that I can't guarantee this will be without issue from a code point of view, from an interaction point of view I still can't see an issue.

In other words, had ms been written today, I guarantee its design would be more ergonomic (with important registers not being overwritten when the package is loaded). The same can be said of the other classical macro packages.

What reason would someone have to specifically override a register on the command line with -r or -d but not want to override it...?

[Alhadis]

What reason would someone have to specifically override a register on the command line with -r or -d but not want to override it...?

Erh, what? You just contradicted yourself…

What I propose is just to switch the order of the code the appends macro files vs command line registers.

Like I said before, this would make it impossible to configure a macro package's behaviour at startup. It might make ms a wee bit more ergonomic to use, but it's going to cripple any macro package that contains code like this:

.if \n[optional-bootstrap-behaviour] \{\
.\" …
.\}

In short, it's a fundamentally useless change solving no real issue whilst introducing ones of its own.

[rnkn]

What reason would someone have to specifically override a register on the command line with -r or -d but not want to override it...?

Erh, what? You just contradicted yourself…

I'm responding here to your assertion that macro files should have "important" registers that the user cannot override:

with important registers not being overwritten when the package is loaded

You're proposing here that the user would want to add command line flags to override these registers but that these should not actually override anything. What purpose would that serve?

Like I said before, this would make it impossible to configure a macro package's behaviour at startup. It might make ms a wee bit more ergonomic to use, but it's going to cripple any macro package that contains code like this:

.if \n[optional-bootstrap-behaviour] \{\
.\" …
.\}

Unfortunately making something bold and italic doesn't change that you're working with faulty assumptions...

So you assume that macro files take optional bootstrap flags in the form of either string or number registers. (Your example requires a number register for the condition to be true.)

Macro files are appended to the source file, so the definition of any register contained therein would occur after your example, so it can't be "bootstrapped" in the source file. That's that scenario out.

So then we have the -r flag where the user might define such a register to provide such a "bootstrap option" (as I did).

But as we've found in #111 and #112, they don't. And as you've stated, the macro files predate the addition of the -r flag, so why would they? So this hypothetical scenario can't happen either. At present, the "crippling" you foretell of cannot occur.

But this "bootstrap" scenario is precisely what I am proposing ought to be possible. We are, after all, in agreement on this. You're just thinking about it backwards, i.e. that a macro will have conditionals that take registers beforehand.

In actuality it works the other way around: macro files define macros, which the user can redefine (with .de) or append (with .am) to afterward.

Now, if we could again come back to the issue at hand. Call it bootstrapping if you like. It can be distilled down to a user who wishes to bootstrap with the following:

converter-tool sourcefile | troff -ms -rPS=12p -rPD=0 -dCF=% | dpost > out.ps

Here the "bootstrap" flags for registers PD and CF are ignored.

Normally the user could redefine these at the top of the source file:

.nr PD 0
.ds CF %

But our source file here is not written in roff. It's coming from STDIN from some converter.

Now, again, the solution could be to rewrite all the macro files to give them the kind of conditionals in your example. Or just change the order in which the command line registers vs macro files are included.

The latter just seems the simpler approach and doesn't appear to give rise to any problems.

In short, it's a fundamentally useless change solving no real issue whilst introducing ones of its own.

It's a mistake to consider something useless just because it doesn't fit with your own use. The world is a plurality.

[rnkn]

Huh, so after reviewing @Alhadis's contributions to the project it seems he was somewhat overstating his familiarity with the codebase, or indeed how roff macros work.

If the project maintainer(s) are happy to leave this issue open for a while, I will (eventually) fashion a PR to address the issue, or at least test the theory that it can be addressed.