Enable file_num parsing in fea config

[jmeyers314]

Allows the telescope to be changed file-by-file in multifile configs. For example, one can add a different zernike perturbation to each output file via something like:

template: ./fiducial_in_focus.yaml

input.telescope.fea:
  extra_zk:
    zk: $[0]*(4+file_num)+[1e-6]
    eps: 0.612
output.dir: $f"zk_{file_num+4:02d}"

This snippet will add some Z4 to file 1, some Z5 to file 2, and so on.

Making this possible basically involved separating the config parsing logic from the telescope assembly logic, making sure the former passes as much as possible through galsim.config.ParseValue and related.

[cwwalter]

I'm a bit confused but what file file_num refers to above.

It is being used to set an output directory and I assume this file is not the per CCD files. Is I for multiple telescope files?

[jmeyers314]

I'm a bit confused but what file file_num refers to above.

Yeah, that's something I learned yesterday. It does increment for every detector, but you can make more complicated configs that track "exposure_num" instead of just file_num by dividing by the number of sensors. Here's a recent snippet I've been playing with:

template:...

eval_variables:
  lwavefront_sensors: [191, 192, 195, 196, 199, 200, 203, 204]
  lscience_sensors: $list(range(189))
  lsensors: $wavefront_sensors  # do just the wavefront sensors
  # lsensors: $wavefront_sensors+science_sensors  # do all the sensors
  izk: "$@file_num//len(sensors)+4"
  iseqnum: $zk+100

input.telescope.fea:
  extra_zk:
    zk: $[0]*zk + [1e-6]
    eps: 0.612

output.dir: $f"intra/zk_{zk:02d}"
output.nfiles: $25*len(sensors)  # 4-28 inclusive

This runs 25 different "exposures" applying a different extra zernike phase to each exposure and placing each result in its own directory. Each exposure produces 8 WF sensor CCDs as configured, but you can see how you'd modify to produce the 189 science sensors or 197 WF+science sensors. In fact, I ran this earlier today on the science sensors to produce 25*189 = 4725 CCD images all from a single invocation of galsim blah.yaml.

[cwwalter]

Ah, I see. OK thanks.

One question: since file_num is a galsim builtin, why do you use the @ (which I thought was for things in the dictionary in the rest of the config file) instead of just using it as a variable like you did in your first example?

Is it related to being in the eval section?

[rmjarvis]

The @ is unnecessary there, but allowed. @file_num means get base['file_num']. But this is one of the things that GalSim automatically makes available in eval statements too. cf. https://github.com/GalSim-developers/GalSim/blob/releases/2.4/galsim/config/value_eval.py#L58

[rmjarvis]

BTW, this would be another mechanism for us to make det_num available. Rather than add it to base['eval_variables'], we could have left it in base and added it to galsim.config.eval_base_variables.

[cwwalter]

OK, I see. Thanks.

Maybe if we have more variables than just det_num, it would be good to have an imsim only equivalent of galsim.config.eval_base_variables we could add things to which could then be added to the galsim one all at once.