Merge qso_grid.py and qso_grid_som.py

[dkirkby]

It looks like these programs are essentially the same, except for reading a different input file, so this issue is to add a command-line option to qso_grid.py so that it can read the new file. This will have the benefit of keeping the simulations of the new templates in synch as new features are added. Note that these programs have already diverged a bit:

cd simulate
git diff 76b36147 HEAD qso_grid.py

@orionsom Many of the changes you made would be unnecessary if you define:

z_grid = np.array([args.zmock])

so that the existing structure for looping over the redshift grid (with only one entry in this case) can be re-used.

[orionsom]

Hi David, Yes, qso_grid_som.py was just a very quick work around to process the quasar mocks we produced and with my very limited knowledge of python we wanted to preserve as much of qso_grid.py as possible. Your suggestion seems to merge the codes elegantly without modifying the original structure and functionality of of qso_grid.py and adding the option of reading in individual mocks with specified redshifts. Is it correct to say that adding the definition g_grid = np.array([args.g_mag]) will allow the use of uniquely specified g (in addition of z) for the individual mocks? Also, can you please confirm if it is correct to presume that stellar templates (specifying z=0) can be passed on to qso_grid.py along with quasar mocks?

[dkirkby]

The same trick should work for g_grid, but taking g & z from cmd-line args means that they are fixed for all simulated spectra, or are you re-running the program for each input spectrum?

Stellar input templates with z=0 should be no problem, but make sure your z_grid only contains the value zero to prevent them being redshifted.

[orionsom]

Thanks. Yes, for fixed z and g for each spectra, the code will need to be looped over the input file (and z and g) list.

[matpieri]

Our goal is to take a list of observed frame spectra (agnostic to QSO/star, but with properties tracked), normalise to match he g-band magnitude and spit out a list of DESIfied observed frame spectra. Its nice that your code can be adapted on a case-by-case basis, but we only need the normalisation and the specsim calling aspects. Perhaps a directed code with the loop included would also be useful.

This brings me to a question I’ve been meaning to ask: should we just be using your brick-making code for this? Does this code meet our requirements of DESIfiing an arbitrarily large number of spectra? I understand that it is designed to perform higher level tests such as the impact of sky subtraction residuals, but if it is sufficient to achieve our more limited goal of QSO identification maybe we should shift focus to this. This would be appealing as it allows the DESIfied spectra to potentially represent a versatile sample.

On 10 Nov 2015, at 17:58, dkirkby notifications@github.com wrote:

The same trick should work for g_grid, but taking g & z from cmd-line args means that they are fixed for all simulated spectra, or are you re-running the program for each input spectrum?

Stellar input templates with z=0 should be no problem, but make sure your z_grid only contains the value zero to prevent them being redshifted.

— Reply to this email directly or view it on GitHub https://github.com/dkirkby/ArgonneLymanAlpha/issues/9#issuecomment-155486001.

[julienguy]

I would like to bring to your attention the fact that we plan to have a code largely inspired from qso_grid.py in the pipeline (i.e. beyond the use of the Lya group only). Please see this link and you can also comment there if you have suggestions to make that code more useful for the Lya group (it could eventually replace qso_grid.py in the future) https://github.com/desihub/desisim/issues/39

[matpieri]

Debopam and I have just been talking and he pointed out that out that qso_grid.py already has grid option. On reflection perhaps we should be using a grid but instead of stepping through g and z, we specify one g and z for each input QSO mock. We then need to parse your code a suite of QSO mocks and an associated (g,z) grid. How does that sound?

On 10 Nov 2015, at 18:15, Matthew Pieri matthew.pieri@lam.fr wrote:

Our goal is to take a list of observed frame spectra (agnostic to QSO/star, but with properties tracked), normalise to match he g-band magnitude and spit out a list of DESIfied observed frame spectra. Its nice that your code can be adapted on a case-by-case basis, but we only need the normalisation and the specsim calling aspects. Perhaps a directed code with the loop included would also be useful.

This brings me to a question I’ve been meaning to ask: should we just be using your brick-making code for this? Does this code meet our requirements of DESIfiing an arbitrarily large number of spectra? I understand that it is designed to perform higher level tests such as the impact of sky subtraction residuals, but if it is sufficient to achieve our more limited goal of QSO identification maybe we should shift focus to this. This would be appealing as it allows the DESIfied spectra to potentially represent a versatile sample.

On 10 Nov 2015, at 17:58, dkirkby <notifications@github.com mailto:notifications@github.com> wrote:

The same trick should work for g_grid, but taking g & z from cmd-line args means that they are fixed for all simulated spectra, or are you re-running the program for each input spectrum?

Stellar input templates with z=0 should be no problem, but make sure your z_grid only contains the value zero to prevent them being redshifted.

— Reply to this email directly or view it on GitHub https://github.com/dkirkby/ArgonneLymanAlpha/issues/9#issuecomment-155486001.

[matpieri]

Interesting. I am happy to be lead by you guys. As I said, the requirement is very simple - astrophysical mock in, DESIfied spectrum out.

I note that you have attached Debopams modification to the qso_gri.py there. Note that this was little more than a quick work around and shouldn’t be considered as a spur to the code development.

On 10 Nov 2015, at 18:23, julienguy notifications@github.com wrote:

I would like to bring to your attention the fact that we plan to have a code largely inspired from qso_grid.py in the pipeline (i.e. beyond the use of the Lya group only). Please see this link and you can also comment there if you have suggestions to make that code more useful for the Lya group (it could eventually replace qso_grid.py in the future) desihub/desisim#39 https://github.com/desihub/desisim/issues/39 — Reply to this email directly or view it on GitHub https://github.com/dkirkby/ArgonneLymanAlpha/issues/9#issuecomment-155497482.

[dkirkby]

The new quickbrick is definitely the way forward for running specsim on a large scale and packaging the results in a standard output format, but it is still useful to refine qso_grid in the near term, as a test bed for ideas that might end up in quickbrick.

We still need to think about how to model the input spectra to quickbrick, including the various effects we worked on at Argonne, and coordinating with other similar efforts, such as the desisim.qso_template module.

[dkirkby]

@matpieri This was a cross-referencing link inserted automatically by github, to alert readers of that other issue that it was being discussed here. There was no code transfer involved.

I note that you have attached Debopams modification to the qso_gri.py there. Note that this was little more than a quick work around and shouldn’t be considered as a spur to the code development.