DESTFNSCRIPT is missing msg['new_file']

MetPX / sarracenia

https://MetPX.github.io/sarracenia

GNU General Public License v2.0

45 stars 22 forks source link

DESTFNSCRIPT is missing msg['new_file'] #774

Closed reidsunderland closed 12 months ago

reidsunderland commented 1 year ago

Many internal plugins were written assuming that msg['new_file'] would exist within a destfn plugin. But destfn plugins are invoked before msg['new_file'] gets set.

This might be not a problem - the developer can use msg['relPath'].split('/')[-1] to get the filename.

But other plugins have msg['new_file'] available, so maybe it's more intuitive if destfn scripts also have new_file.

@petersilva is looking at related parts of the code for #772.

petersilva commented 1 year ago

yeah... so ... the issue is that DESTFNSCRIPTS entry points are being called in the routines that set up msg['new_dir'] and msg['new_file'] for after_accept entry points that run later... so it's kind of a inception issue where msg['new_file'] should be set before DESTFNSCRIPTS is called, but DESTFNSCRIPTS is called to help set msg['new_file']...

a couple of different approaches as possible...

set msg['new_file'] with "warmup" values before the call. to sundew_getDestInfos() (the routine that calls the script.)
more that call later in the routine...

but the thing is, msg['new_file'] is set with the return value from that call when it is active... hmm... ponder a bit.

petersilva commented 1 year ago

I have a fix, but it is a bit terrifying...

I moved the call to the very end of the getDestInfos ... after updatePaths has run, so it has the final values.
Then I update the variables that such routines should be messing with (new_file and new_relPath) based on output.
it passes the flow tests, the hpc_mirror tests, and I tried the broken desftnscript we played with together. It seems ok in all cases.

I'm worried that there is a reason is was where it was in the sequence. This is reversing the order of calls to getDestInfos and set_dir_pattern (v2 name... sr3 is: sundew_dirPattern ) I worry that the dirPattern thing uses stuff set in destfnscript somehow... now it looks like destfnscript is just about filenames, and dirPattern is about setting the new_dir (the directory the file will end up in.) ... so ... looks ok.