Better determination of output files.

matz-e commented 6 years ago

A rough draft for a script to automatically determine outputs and global tags from psets that require a release other than the current active one. Somehow #634 triggered me to do this, since this has been one of my major pet-peeves all along. Since I'm currently not able to test this, additional work may be required, so maybe this is just a dead end? Feel free to built on top or discard!

This is also just a quick hack; Ideally, there would be a dedicated cmssw.workflow module/class that takes care of this in a cleaner way.

Maybe fixes #317 and maybe fixes #634. Additional changes may have to be applied.

Please make sure that the following items are taken care of if needed:

[x] Has the database format changed? (renamed or new columns, tables) Or did any of the project layout change? (files required to run the workflows)
- Please increase the version number VERSION after the imports in lobster.util. This will ensure that Lobster does not try to load old projects with an incompatible version.
[x] Did the required Work Queue version change?
- Please update the three version numbers in doc/install.rst. Adjust the tarball name in the install_dependecies.sh script, too.
[x] Are all additional dependencies in setup.py?
[x] Mark any issues as closed either in commits or in this pull request.

klannon commented 6 years ago

Thanks for putting this together. I can take a look at this when I’m back from vacation.

klannon commented 5 years ago

Hey @matz-e! So after too long a time of ignoring this PR, I actually got around to testing it. There were a number of bug fixes I had to add to make it work. I'd like to commit my fixes and then merge this PR, but I don't seem to have permission to push to your repo any more. Alternatively, I could just push my version to a new branch in the NDCMS lobster repo, close this PR, and carry on from there. However, I'd prefer to do it through this PR since you're the one who got it started. Any suggestions?

matz-e commented 5 years ago

Hi @klannon, that's great! I re-forked this repo after we moved it into NDCMS, I've just re-added you as a collaborator on my fork. Although according to the little checkbox on the PR, you should be able to push to my autosense branch if you're a Lobster maintainer…

If you have any questions about the PR, I can try to recollect my thoughts about it and will answer them as best as possible!

klannon commented 4 years ago

I've been waiting to do more testing on this, but it fixes an important problem that people sometimes run into, so I'm just going to go ahead and merge it and we'll fix the problems that haven't been caught yet when they're reported.

NDCMS / lobster

Better determination of output files. #635