Open jessmuir opened 2 months ago
Note that this isn't really a bug: one could argue that it's better to avoid using this bash variable in the original ini file, and then use a cosmosis campaign component to overwrite the scale cuts instead of changing what scale cut ini file is included. I mainly just wanted to flag the fact that this behavior changed.
On a related note: for a different element of the same project I've written a couple utility functions that generate a cosmosis-campaign component for a yml file given scale cuts specified in an ini file, and vice versa. I don't know if there's a sensible way to incorporate that into the campaign structure, but since testing how runs respond to different scale cuts is a somewhat common test, and since transferring scale cut info between ini and yml formats is fairly tedious (and risks of typos), others might find this automation useful. Let me know if there's a helpful way to share this.
Hi Jessie - thanks for flagging this. I made a change so that campaign environment variables were only supported in values, not in keys or section names. I had thought that was the best idea, but it does mean that there is an inconsistency between the behaviour when using campaigns and when using the original ini files, which certainly isn't good. But at least npthing should silently go wrong.
I think I'd have to completely rewrite how the runs are stored in order to make this work while still allowing child runs to override the environment variables of their parents. I think I'd like to do that at some point in order to make Runs and Campaigns more usable from scripts, but that will take some time, and I think you're right that ultimately it would be better to avoid env vars here (and indeed everywhere) in favour of using components.
So I'm not sure what to do here. Any thoughts welcome!
I think the change is reasonable, and if having both behaviors (being able to specify filenames with env variables as well as letting children override parents' env) isn't easy to implement.
It'll require people to make some changes to workflow in terms of how people often set choices for 2pt scale cuts, and thus will mean a bit more legwork for folks transiting from existing lots-of-related-ini-fiels setups to campaign setups. That's not a show stopper, but could maybe be worth adding a note about it to the campaign setup documentation?
I recently updated cosmosis to version 3.8 and have encountered a new error when using env variables with cosmosis campaign.
The yml and ini files I'm working with ran ok before I updated cosmosis. Unfortunately I'm not sure what the previous version was before I updated to 3.8, so I'm not sure exactly what change interfered with this.
My setup is that I have an ini files specifying a pipeline, where a 2pt_like module definition includes the line
That bash varialbe is specified in a run definition in the campaign file, using
When run cosmosis-campaign, either attempting to do a test run or looking at available runs using cosmosis-campaign -l, I get the error:
Perhaps a recent update to ini file parsing changed this behavior? As I said, this setup didn't cause problems before yesterday when I updated cosmosis?