Inserting pdf_colorstack whatsits from Lua causes a failure at shipout

[gucci-on-fleek]

Inserting pdf_colorstack whatsit nodes from Lua causes a compilation failure when the page is shipped out. Both of these examples:

\input ltluatex

\directlua{
luatexbase.add_to_callback("post_linebreak_filter", function(head)
    local start_colour = node.new("whatsit", "pdf_colorstack")
    start_colour.stack = 0
    start_colour.command = 1
    start_colour.data = "1 0 0 rg"

    local end_colour = node.new("whatsit", "pdf_colorstack")
    end_colour.stack = 0
    end_colour.command = 2

    local last_line = node.tail(head).list
    start_colour.next = last_line
    node.tail(last_line).next = end_colour
    node.tail(head).list = start_colour

    return true
end, "test")
}

\input knuth

\bye

\input ltluatex

\directlua{
    local start_colour = node.new("whatsit", "pdf_colorstack")
    start_colour.stack = 0
    start_colour.command = 1
    start_colour.data = "1 0 0 rg"
    node.write(start_colour)
}

\input knuth

\directlua{
    local end_colour = node.new("whatsit", "pdf_colorstack")
    end_colour.stack = 0
    end_colour.command = 2
    node.write(end_colour)
}

\bye

work with LuaTeX, but fail in luametaplain with the following error:

{/usr/local/texlive/2022/texmf-var/fonts/map/pdftex/updmap/pdftex.map}This is LuaMetaTeX, Version 2.10.01 
(lmtx-colorstack.tex(ltluatex.tex)(knuth.tex)
! registered function call [65589]: ...f/tex/luametatex/luametalatex/luametalatex-firstcode.lua:102: attempt to index a nil value (local 'p')
stack traceback:
    ...f/tex/luametatex/luametalatex/luametalatex-firstcode.lua:102: in local 'handler'
    .../tex/luametatex/luametalatex/luametalatex-nodewriter.lua:688: in field '?'
    .../tex/luametatex/luametalatex/luametalatex-nodewriter.lua:353: in field '?'
    .../tex/luametatex/luametalatex/luametalatex-nodewriter.lua:381: in field '?'
    .../tex/luametatex/luametalatex/luametalatex-nodewriter.lua:381: in field '?'
    .../tex/luametatex/luametalatex/luametalatex-nodewriter.lua:381: in field '?'
    .../tex/luametatex/luametalatex/luametalatex-nodewriter.lua:781: in function 'luametalatex-nodewriter'
    ...mf/tex/luametatex/luametalatex/luametalatex-back-pdf.lua:113: in function <...mf/tex/luametatex/luametalatex/luametalatex-back-pdf.lua:97>.
<macro> \now@and@everyjob 
    \shipout \vbox {\makeheadline \pagebody \makefootline }
    \advancepageno \ifnum \outputpenalty >-\@MM \else \dosupereject \fi 
<output> 
    {\plainoutput 
    }
<macro> \now@and@everyjob 
    \par \penalty -\@MM 
<macro> \now@and@everyjob 
    \par \vfill \supereject 
    \end 
<line 1.25> 
    \bye
? 
)</usr/local/texlive/2022/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb>
(see the transcript file for additional information)
 node memory still in use:
  16 hlist, 5 vlist, 1 rule, 23 gluespec, 178 nil, 3 par, 52 disc, 149 glue, 638 glyph, 8 penalty, 52 kern, 6 open
Output written on lmtx-colorstack.pdf (2 pages, 4739 bytes).
Transcript written on lmtx-colorstack.log.

I'm also open to using a different method to colour a node list if this method is unsupported in LuaMetaLaTeX/LuaMetaPlain.

[zauguin]

Instead of start_colour.stack = 0 etc. you need node.setwhatsitfield(start_colour, 'stack', 0) and similar in LuaMetaLaTeX. That's needed for all whatsit fields.

(Untested, I'll write a proper answer when I have more time)

[gucci-on-fleek]

Instead of start_colour.stack = 0 etc. you need node.setwhatsitfield(start_colour, 'stack', 0) and similar in LuaMetaLaTeX. That's needed for all whatsit fields.

Unfortunately, that doesn't seem to solve the problem. It does at least give a different error message. I was able to fix that error message by using 28 instead of "pdf_colorstack" as the subtype, then I got a different error that I was able to fix by setting the colorstack` field to a specific table.

Running this code:

\input ltluatex

\directlua{
luatexbase.add_to_callback("post_linebreak_filter", function(head)
    local start_colour = node.new("whatsit", 28)
    node.setwhatsitfield(start_colour, "colorstack", {
        page_stack = {},
        mode = "page",
        action = "push",
        data = "1 0 0 rg"
    })

    local end_colour = node.new("whatsit", 28)
    node.setwhatsitfield(end_colour, "colorstack", {
        page_stack = {},
        mode = "page",
        action = "pop"
    })

    local last_line = node.tail(head).list
    start_colour.next = last_line
    node.tail(last_line).next = end_colour
    node.tail(head).list = start_colour

    return true
end, "test")
}

\input knuth

\bye

works without any errors, but it doesn't actually put any colour on the page. I'm pretty sure that page_stack shouldn't just be an empty table, but I'm not exactly sure what it should be.

[zauguin]

Yes, the interface for pdf_colorstack was not all that compatible with LuaTeX from the Lua side. I made them a bit more similar, so now you can do

\input ltluatex

\directlua{
    local start_colour = node.new("whatsit", node.subtype'pdf_colorstack')
    node.setwhatsitfield(start_colour, 'stack', 0)
    node.setwhatsitfield(start_colour, 'command', 1)
    node.setwhatsitfield(start_colour, 'data', "1 0 0 rg")
    node.write(start_colour)
}

\input knuth

\directlua{
    local end_colour = node.new("whatsit", node.subtype'pdf_colorstack')
    node.setwhatsitfield(end_colour, 'stack', 0)
    node.setwhatsitfield(end_colour, 'command', 2)
    node.write(end_colour)
}

\bye

[gucci-on-fleek]

Works perfectly, thanks!