About runcmd, 2 feature requests, one comment and one issue

[jlaurens]

If checkformat is not "context", runcmd defines the TEXMFCNF environment variable to be .:. Two requests on that matter:

1. If I understand correctly, this is related to section 2.9 of the documentation titled "Settings in texmf.cnf" which is rather imprecise. I suggest to replace the first sentence with "To allow application of non-standard TEX trees or similar non-standard settings, l3build sets the environment variable TEXMFCNF in runcmd to first read a custom texmf.cnf file put in the support folder, if any."
2. When the non-standard settings are not intrinsically tied to the project but to the configuration of the host machine, using a custom texmf.cnf file put in the support folder does not make sense. Instead, setting the TEXMFCNF environment variable is the way to go. Unfortunately, runcmd definitely overrides TEXMFCNF. There is indeed a workaround by redefining runcmd in build.lua (what I actually use) but this is far from efficient coding. If TEXMFCNF is already set in the environment, there are 2 possibilities: either leave TEXMFCNF as it is or do the equivalent of export TEXMFCNF=$TEXMFCNF:.:. I would go for the former assuming that people using TEXMFCNF environment variable know what they are doing and will want full control.

One comment: why not match(checkformat, "^context$") instead of just checkformat~="context" on line 193?

One issue on line 207, there is a missing .. os_concat at the end, actually things cannot work properly when checkformat is "context" and vars is not empty.

[josephwright]

One comment: why not match(checkformat, "^context$") instead of just checkformat~="context" on line 193?

That is I think historical: we had a more complex match earlier. I'll adjust.

[josephwright]

One issue on line 207, there is a missing .. os_concat at the end, actually things cannot work properly when checkformat is "context" and vars is not empty.

I'm not sure I'm looking at the right lines: could you quote the surroundings?

[jlaurens]

Here is the link https://github.com/latex3/l3build/blob/e2060d6730230235e2b81f6db9199b40fa43e317/l3build-aux.lua#L207

[josephwright]

Here is the link

https://github.com/latex3/l3build/blob/e2060d6730230235e2b81f6db9199b40fa43e317/l3build-aux.lua#L207

But that only applies if env is empty as there is no concatenation to do

[josephwright]

2. When the non-standard settings are not intrinsically tied to the project but to the configuration of the host machine, using a custom texmf.cnf file put in the support folder does not make sense. Instead, setting the TEXMFCNF environment variable is the way to go. Unfortunately, runcmd definitely overrides TEXMFCNF. There is indeed a workaround by redefining runcmd in build.lua (what I actually use) but this is far from efficient coding. If TEXMFCNF is already set in the environment, there are 2 possibilities: either leave TEXMFCNF as it is or do the equivalent of export TEXMFCNF=$TEXMFCNF:.:. I would go for the former assuming that people using TEXMFCNF environment variable know what they are doing and will want full control.

I'm wondering what situation you are imaging here: the entire point of resetting the texmf setup is we want to be sure there is no non-standard stuff going on.

[jlaurens]

@josephwright If env is empty and vars only contains variable FOO, then line 210

return run(dir,set_epoch_cmd(epoch, forcedocepoch) .. env .. cmd)

is equivalent to

return run(dir,set_epoch_cmd(epoch, forcedocepoch) .. os_setenv .. " " .. 'FOO' .. "=" .. envpaths .. cmd)

as envpaths does not end wit os_concat, the content of the cmd variable is considered the continuation of envpaths.

[jlaurens]

Concerning point 2) I perfectly understand your explanation and it makes sense for LaTeX. But for other packages, the testing policy might be different and the developer might want to use the TEXMFCNF variable differently. In a test driven development process, developers may want to test against a local unofficial texlive/2025/dev distribution. The location of this auxiliary distribution depends on the developer and should not be part of the project and also should not be saved in a custom texmf.cnf that would be tracked by a versioning system.

[josephwright]

Concerning point 2) I perfectly understand your explanation and it makes sense for LaTeX. But for other packages, the testing policy might be different and the developer might want to use the TEXMFCNF variable differently. In a test driven development process, developers may want to test against a local unofficial texlive/2025/dev distribution. The location of this auxiliary distribution depends on the developer and should not be part of the project and also should not be saved in a custom texmf.cnf that would be tracked by a versioning system.

I'm still not seeing how that links to changing your texmf settings: you'd just pick the right path ... which is what I do for pre-tests, etc.

[josephwright]

@josephwright If env is empty and vars only contains variable FOO, then line 210

return run(dir,set_epoch_cmd(epoch, forcedocepoch) .. env .. cmd)

is equivalent to

return run(dir,set_epoch_cmd(epoch, forcedocepoch) .. os_setenv .. " " .. 'FOO' .. "=" .. envpaths .. cmd)

as envpaths does not end wit os_concat, the content of the cmd variable is considered the continuation of envpaths.

Ah, right, got it: adjusted

[josephwright]

Concerning point 2) I perfectly understand your explanation and it makes sense for LaTeX. But for other packages, the testing policy might be different and the developer might want to use the TEXMFCNF variable differently. In a test driven development process, developers may want to test against a local unofficial texlive/2025/dev distribution. The location of this auxiliary distribution depends on the developer and should not be part of the project and also should not be saved in a custom texmf.cnf that would be tracked by a versioning system.

I'm still not seeing how that links to changing your texmf settings: you'd just pick the right path ... which is what I do for pre-tests, etc.

Also, I'm not sure how one can handle a local texmf.cnf for a project and the possibility that the developer also has a custom setup: only one can 'win'.

[jlaurens]

What do you mean by ´you just pick the right path'? In the above mentionned situation there are at least 2 right paths, and developers will want to handle them concurrently.

[josephwright]

@jlaurens For example, on my Windows system I have TL'09 to TL'24 plus MiKTeX installed. In my PATH, I have just TL'24. So from the Command Prompt when I do l3build <whatever> the files picked up are from TL'24. If I want to test a different version, I do at a Command Prompt set PATH=<some-other-TeX-root>;%PATH% and then that prompt session will use a different TeX system. So with two Command Prompts open I can run TL'24 and some other system in parallel. Same idea applies on my macOS setup, or indeed any OS at all.

[jlaurens]

@josephwright What you describe is the "normal" setup for everyday use, but it is not completely suitable for development. First, it is based on the assumption that the TeX distributions are stable, in particular that the default location of the TeXMF tree, which is hard coded in the various engines through kpathsea, is the right one. During development, there are many problems with this setup

1. In testing driven development, you must switch back and forth between different consoles, which is inefficient in terms of time and cognitive pressure
2. automate testing against different distributions is fairly complicated and might depend on the OS
3. this does not simply allow to separate the binaries from the texmf tree, you can only pickup the texmf tree location that is hard coded in the engine

I have in mind 3 situations

1. A new feature is added to a package, automated testing will at least use a TL'25 development version containing the new package and a custom TL'24 updated with the new package version (with tlmgr for example).
2. The project is a new LaTeX package which implementation does not use cutting edge technologies. It will be available for TL'25 users as well as TL'<25 users. Automated testing will at least use a TL'25 development version containing the new package and a freezed TL'24 with the new package in some local texmf tree dedicated to development.
3. The development of new features of one engine: the PATH will give access to this engine binaries wherever it is located and texmf.cnf will do the rest.

[josephwright]

@jlaurens I'm still not really clear on the use case, but lets look at something that will work for you :) We can adjust the Lua - the question is what would you expect here?