nside in spectra files - Githubissues

desihub / desispec

DESI spectral pipeline

BSD 3-Clause "New" or "Revised" License

36 stars 23 forks source link

nside in spectra files #567

Open ngbusca opened 6 years ago

ngbusca commented 6 years ago

Is the nside that is used to produce the spectra files logged anywhere other than the path to the files (e.g. in spectra-64)? We (myself, @londumas and others) are trying to find a way to determine in which spectra file a given target will have its spectrum given the RA, DEC (or the TARGETID for that matter), and it seems impossible unless one knows the value of nside or loops over all the files.

sbailey commented 6 years ago

Parsing spectra-64 to get nside=64 is a reliable method, though it does seem odd to have to parse a directory name to learn the nside.

PR #566 restored HPXNSIDE and HPXPIXEL keywords to the output spectra file from the standard pipeline, thought we still need to verify / update for quickspectra and quicklya scripts. But even then you would need to read one file to learn the nside to use when searching all the rest.

defacto using a default nside=64 and making that a input parameter for your code would also work.

Do you have an suggestion for where you think it should be kept for end-users to discover the nside?

ngbusca commented 6 years ago

Ah, I didn't realize it's written in the spectra files themselves! (it's not listed in readthedocs, but I totally understand if that's not up-to-date). I'm ok with reading one spectra file to find out. If you stored it in another file I would still have to read that other file, and would have to keep track of where that other file is.

ngbusca commented 6 years ago

I'm reopening this issue since the latest sim files produced by @julienguy using quickquasar don't have these keywords (dc17b files do have the slightly different NSIDE and HPIX).

ngbusca commented 6 years ago

Two more comments for completeness:

nside produced from quickquasar is 16, so hardcoding doesn't work
parsing the path works for nside but we still need the healpix scheme (NEST is fashionable these days, but maybe in 42 years we forget this choice).

julienguy commented 6 years ago

quickquasar is using exactly the same pixelization as the mock input .
I can modify quickquasar to add keywords, but I don't know the scheme used for the mocks. so I can't set a HPXNEST keyword .
I see the spectra format was modified in desispec.scripts.group_spectra but not in desispec.io.write_spectra which is the routine used in desispec.scripts.quickspectra. I can pass those header keywords via the meta argument so that's fine for now, but having two writing routines will pose problems in the future. I know the desispec.scripts.group_spectra I/O are much faster thought.

So, a single question: what is the healpix scheme in the current lya mocks?

sbailey commented 6 years ago

I sure hope that the mocks also used NESTED. Let's use that everywhere as if RING doesn't exist. It will be chaos if some places use one and others use the other.