VIDA-NYU / reprozip

ReproZip is a tool that simplifies the process of creating reproducible experiments from command-line executions, a frequently-used common denominator in computational science.
https://www.reprozip.org/
BSD 3-Clause "New" or "Revised" License
305 stars 34 forks source link

Old standalone build of tracer quits abruptly... any idea why? #387

Closed yarikoptic closed 1 year ago

yarikoptic commented 1 year ago

in #386 we reported on "our" use of reprozip failing to deal with reprozip tracer output. As we looked deeper, we recalled that we are using a tracer which we "extracted" from reprozip which we "extracted" from reprozip - just to build self-sufficient statically linked rztracer, with minimal diff within the branch: https://github.com/ReproNim/reprozip/tree/extract-c-tracer/reprozip/native (branched off at 9f0162347eb67fd7a538a2b8def29329f1ef9ffe) . The tracer worked years back but now when we got back to reincarnate that project with @asmacdo we found that it is

❯ ./rztracer --help
Usage: rztracer [--logfile FILE] [--dbfile FILE] -- CMD [ARGS ...]
❯ ./rztracer --logfile logfile --dbfile dbfile -- ls
[REPROZIP] 09:26:48.574 CRITICAL: cleaning up, 1 processes to kill...
❯ echo $?
0

and dbfile is empty... no expected output from ls, so it doesn't work and I wondered if that CRITICAL is an error message and what could lead to such behavior of reprozip's tracer?

NB trying to merge in the changes since then shows that tracer now doesn't have its own logging, uses python logging and that we would love to avoid to keep the tracer minimalistic, and I've not tried to replace logging with old fashion one yet. But may be you recall some critical changes to tracer you had to do since back then?

remram44 commented 1 year ago

It's a bit hard to tell. Can you perhaps run with higher verbosity?

If you could check in the updated forked code, I could take a look.

remram44 commented 1 year ago

I'm going to close this for now but let me know if there's something I can look at!

asmacdo commented 1 year ago

thanks @remram44 , we're digging in today. If we need it, what's the preferred venue for getting more help, reopen here or slack or something?

remram44 commented 1 year ago

Here is good, otherwise we have a Matrix room

yarikoptic commented 1 year ago

Just FTR, using reprozip of that old version which we based on, also resulted in the similar crash:

❯ reprozip trace -d /tmp/reprozip-trace-`git describe` ls
[REPROZIP] 13:43:45.275 CRITICAL: cleaning up, 1 processes to kill...
Traceback (most recent call last):
  File "/home/yoh/proj/repronim/reprozip/reprozip/reprozip/main.py", line 425, in main
    status = args.func(args)
  File "/home/yoh/proj/repronim/reprozip/reprozip/reprozip/main.py", line 237, in trace
    status = reprozip.tracer.trace.trace(args.cmdline[0],
  File "/home/yoh/proj/repronim/reprozip/reprozip/reprozip/tracer/trace.py", line 341, in trace
    c = _pytracer.execute(binary, argv, database.path, verbosity)
_pytracer.Error: Error occurred
❯ reprozip --version
reprozip version 1.1.0
❯ git describe
1.0.12-96-g0497b229

so indeed tracer was fixed since then. So we would need to update our "standalone" tracer, although would need to figure out how to deal with the code now using python's logger (not that we care about logging much besides troubleshooting -- may be should just strip it ).

remram44 commented 1 year ago

It's pretty self-contained so you just need to provide the functions from log.h (and delete pylog.c). You should be able to adapt the old log.c, the only difference is that the "tag" argument was removed.