cosmosis-campaign: updating env variables compared to parent run

[jessmuir]

This is another issue that I submitted about cosmosis-campaign over on the standard library repo by mistake. I initially opened the issue last week, I'm copying the text over to get it on the correct git repo. Apologies for the mess!

---

Hi Joe!

I've just run into something with cosmosis-campaign that seems worth flagging. In my workspace, as we've pretty commonly done in DES cosmosis set-ups, I've got a bash variable ${DATA_VECTOR} which is then used to set the option in the [DEFAULT] section, 2PT_FILE = data_vectors/${DATA_VECTOR}.

Without campaigns, we've typically done this and set up the fits file name as part of the job script or some other bash script specifying the cosmosis run. In my campaign setup I'm trying to move away from this. I notice that there's an env option to set a dictionary of environment variables for a run in the yaml file. This works for setting the env for a baseline run, but if I try to change the variable in another run that has that as a parent, it doesn't change it in the pipeline.

In other words, doing something like this DOES NOT work:

runs:
  - name: baseline
    base: params.ini
    env:
      DATA_VECTOR : fiducial_2pt_data.fits

  - name: alt_data
    parent: baseline
    env:
      DATA_VECTOR : alternative_2pt_data.fits

That is to say, if I do a test run with alt_data, there is no change to the likelihood compared to baseline, and it still runs with no errors if I set the alt_data env variable DATA_VECTOR to a nonexistent filename.

Two things that do seem to work are either doing:

  - name: alt_data
    parent: baseline
    env:
      DATA_VECTOR : alternative_2pt_data.fits
    params:
    - DEFAULT.2PT_FILE = data_vectors/${DATA_VECTOR}

or

  - name: alt_data
    parent: baseline
    params:
    - DEFAULT.2PT_FILE = data_vectors/alternative_2pt_data.fits

This suggests the issue has something to do with the order in which files are loaded and environment variables are translated. As I noted above, I'm able to get the desired behavior, so this isn't a bug perse. That being said, it might be worth flagging this or including some warning in documentation about the env: yaml options.

[joezuntz]

Looking at this in branch https://github.com/joezuntz/cosmosis/tree/campaign-env-fix

[joezuntz]

Fixed in #116