Create log file for failed builds

[StevenClontz]

Close this if I'm just forgetting where to look but we should provide a full log file for failed builds.

[rbeezer]

And I'd like to use some Python package (warnings?) to be more rigorous about this.

So if you decide on infrastructure, I'd be game to work on pretext\pretext to gradually clean this up beyond my home-grown routines.

And maybe messages originating in stylesheets can be incorporated? They could take any form (meaning any text is possible).

[StevenClontz]

Leaving this here in case it helps: https://stackoverflow.com/questions/45285601/how-to-get-more-info-from-lxml-errors

[StevenClontz]

(aside for @rbeezer: some day I'll return to PTX meetups... after the 3rd hurricane of the year hit us this week, I'm finding little time to do much beyond what's absolutely required of me recently)

[rbeezer]

Take a look at the stanza here:

https://github.com/rbeezer/mathbook/blob/fd5c4813170cbb869d18894773871ea5888796d2/pretext/pretext.py#L1400

(No problem - sorry to hear about the wind and rain. We'll see you when life returns to something like normal.)

[StevenClontz]

Idea: at the end of cli.main(), find a hook that runs after everything else to write the logger to a file in .logs/timestamp.txt.

[bnmnetp]

One thing that would really help the user experience for a PreTeXt build would be to have some intermediate messages to let one know where in the process we are. Right now I just start the command and then wait -- a long enough time to get anxious -- and then finally a bunch of messages spew out and I'm done.

Just as a point of contrast a build of RST with sphinx lets me know what file its reading and in the end what files its writing. Error messages pop out at the time they are encountered. This gives me a much better sense of what is going on and approximately how fast things are going and how much time I have to wait before the build is complete.

[StevenClontz]

@rbeezer thoughts?

See https://lxml.de/xpathxslt.html#errors-and-messages - it's unclear to me that we can run transform ansynchronously, and if so, that we could capture a stream of messages in the error log. This may be a limitation of LXML (which perhaps isn't a limitation of xsltproc on the command line).

[rbeezer]

Sort of like here:

https://github.com/PreTeXtBook/pretext/blob/35840493c1d403bd30ed8dbcf60ffab1a41f4b6c/pretext/pretext.py#L2245-L2267

Different in what way?

[rbeezer]

Maybe I should review the whole thread? Yes, I'd like messages as they come, a reason I sometimes just use xsltproc. No good ideas off the top of my head about how we can get these on-the-fly.

[StevenClontz]

One solution (part of core) look for xsltproc and use it if it's available, with LXML as the fallback.

Another (goofy?) solution: try to contribute to LXML an option for the transform object that streams messages to output live.

On Tue, May 17, 2022, 1:20 PM Rob Beezer @.***> wrote:

Maybe I should review the whole thread? Yes, I'd like messages as they come, a reason I sometimes just use xsltproc. No good ideas off the top of my head about how we can get these on-the-fly.

— Reply to this email directly, view it on GitHub https://github.com/PreTeXtBook/pretext-cli/issues/67#issuecomment-1129179763, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAL4YUDXXC7GDNFF3P4IXZ3VKPPPLANCNFSM4S73HLJA . You are receiving this because you authored the thread.Message ID: @.***>

[bnmnetp]

Not sure if lxml works with threads but maybe a solution would be to run transform in its own thread and then have another thread check/output the error log?? I'm kind of wildly speculating since I've spent about 1 hour with lxml code.

[StevenClontz]

The problem is that I believe the error log doesn't exist in any accessible way until xslt() (or whatever you call the object) is finished and has returned the resulting object with an error_log attribute.

https://github.com/PreTeXtBook/pretext/blob/master/pretext/pretext.py#L2247

[bnmnetp]

Take a closer look at the line below:

 result_tree = xslt(src_tree, **stringparams)

The object you are waiting for is result_tree, while the error_log is actually coming from the xslt object.

I put together this quick prototype using a thread and tried it out. I have tried it on a couple of books and It begins printing out messages from the transform right away!

       result_tree = []

        def transform():
            global result_tree
            result_tree = xslt(src_tree, **stringparams)

        parse_t = Thread(target=transform)
        parse_t.start()
        done = False
        start = 0
        while not done:
            parse_t.join(0.5)
            still_going = parse_t.is_alive()
            if still_going:
                end = len(xslt.error_log)
                for line in range(start, end):
                    print(xslt.error_log[line])
                start = end
            else:
                done = True

Would be happy to clean it up a bit and submit as a PR.

[StevenClontz]

🎉 looks slick!

On Thu, May 19, 2022, 7:53 AM Bradley Miller @.***> wrote:

Take a closer look at the line below:

result_tree = xslt(src_tree, **stringparams)

The object you are waiting for is result_tree, while the error_log is actually coming from the xslt object.

I put together this quick prototype using a thread and tried it out. I have tried it on a couple of books and It begins printing out messages from the transform right away!

   result_tree = []

    def transform():
        global result_tree
        result_tree = xslt(src_tree, **stringparams)

    parse_t = Thread(target=transform)
    parse_t.start()
    done = False
    start = 0
    while not done:
        parse_t.join(0.5)
        still_going = parse_t.is_alive()
        if still_going:
            end = len(xslt.error_log)
            for line in range(start, end):
                print(xslt.error_log[line])
            start = end
        else:
            done = True

Would be happy to clean it up a bit and submit as a PR.

— Reply to this email directly, view it on GitHub https://github.com/PreTeXtBook/pretext-cli/issues/67#issuecomment-1131651854, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAL4YUB7M3VFXMMBZAUXYLLVKY2TJANCNFSM4S73HLJA . You are receiving this because you authored the thread.Message ID: @.***>

[rbeezer]

Would be happy to clean it up a bit and submit as a PR.

+1

[StevenClontz]

Bonus points if we can pass a function to use in place of print (CLI uses the logging module).

[bnmnetp]

That would require another parameter to be added to the xsltproc function. I could add an outputfn parameter and make it default to print so would be fully backward compatible.

[StevenClontz]

That's the pattern I've used before. But I thought about it more and maybe it's out of scope for now - even if you did expose that functionality in Rob's xsltproc, we'd have to add hooks to every higher-level in Rob's pretext.py as well. I'm okay with kicking that can down the road a bit.

[bnmnetp]

If it was me I would want that to be a separate PR anyway.

[rbeezer]

Can we right now do the following?

• Add an outputfn variable inside of core xsltproc().
• Set outputfn = print.
• Code everything to use outputfn.
• Later, make outputfun a parameter, and expand all the higher-level calls in a coordinated fashion?

[StevenClontz]

That wouldn't offend me, but I'm not certain it's much more trouble to find/replace print with outputfn down the road when we use it (and improves readability in the short term if we're just using print).

[bnmnetp]

I think that is exactly the thing as changing the signature of xsltproc to simply add a new parameter outputfn=print. If the named parameter is not supplied it defaults to print. And as you said changing print to outputfn inside xsltproc is easy and will use print by default if the parameter is there.

@rbeezer I can add that to my PR if you want.

[rbeezer]

Right, I forgot about optional/default parameters. As you've noticed, my Python is rudimentary.

I can add that to my PR if you want.

Yes, please! Just push a new commit, I'll consolidate as part of reviewing/merging. I'll wait for that, and may get to it after putting out all the fires today.

[bnmnetp]

Done and pushed.

[StevenClontz]

Closed by 8f5fb8b2a5fd6038fcf4276f2808c18f5747ee63

Heads up @oscarlevin I edited some gitignore files, so that might hit a conflict with your ongoing work.

Currently core Python is printing, not logging, so that output isn't captured. Opened discussion at https://groups.google.com/g/pretext-dev/c/o0jNAaQHBus to fix this.