Latest picca version fails to run on eboss mocks

[moonlovist]

The latest version of picca_delta.py fails to run on eboss mocks, for both desi mode and desi_mocks mode. The following error message appears when running picca_delta.py:

Reading /global/project/projectdirs/desi/mocks/lya_forest/develop/saclay/v4.7/v4.7.01/eboss-0.2/spectra-16//30/3071/spectra-16-3071.fits determining nside nside = 4 -- mean #obj per pixel = 1328.3157894736842 INFO: Adding DLAs Reading DLA catalog from: /global/project/projectdirs/desi/mocks/lya_forest/develop/saclay/v4.7/v4.7.01/eboss-0.2/zcat_desi_drq_DLA_sup_203.fits Traceback (most recent call last): File "/global/homes/t/tanting/.local/cori/3.8-anaconda-2020.11/lib/python3.6/site-packages/fitsio/hdu/table.py", line 1548, in _extract_colnum colnum = self._colnames_lower.index(colstr.lower()) ValueError: 'targetid' is not in list

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/global/homes/t/tanting/picca/bin/picca_deltas.py", line 1025, in main(cmdargs) File "/global/homes/t/tanting/picca/bin/picca_deltas.py", line 602, in main dlas= io.read_dlas(args.dla_vac, obj_id_name='TARGETID') File "/global/homes/t/tanting/picca/py/picca/io.py", line 56, in read_dlas cat = {col: hdul['DLACAT'][col][:] for col in columns_list} File "/global/homes/t/tanting/picca/py/picca/io.py", line 56, in cat = {col: hdul['DLACAT'][col][:] for col in columns_list} File "/global/homes/t/tanting/.local/cori/3.8-anaconda-2020.11/lib/python3.6/site-packages/fitsio/hdu/table.py", line 1605, in getitem return TableColumnSubset(self, res) File "/global/homes/t/tanting/.local/cori/3.8-anaconda-2020.11/lib/python3.6/site-packages/fitsio/hdu/table.py", line 1871, in init self.colnums = [fitshdu._extract_colnum(columns)] File "/global/homes/t/tanting/.local/cori/3.8-anaconda-2020.11/lib/python3.6/site-packages/fitsio/hdu/table.py", line 1550, in _extract_colnum raise ValueError(mess) ValueError: column name 'TARGETID' not found (case insensitive)

[alxogm]

OK, I think we should decide here if we want to have an eboss_mock mode or treat all quickquasars mocks under the desi_mock. If the second case, then my previous PR was not fully backwards compatible clearly... I think soon others might face the same issue so I can fix it. I just need to know if @iprafols @Waelthus agrees with all quickquasar mocks falls into the desi_mocks umbrella, probably best to name it desisim_mocks, not just desi_mocks

[iprafols]

Hi @moonlovist, @alxogm,

Is the problem still open? If so we should address this on the next picca task force telecon. It is not clear not me how to proceed. While it makes sense to have a desisim_mocks mode maybe it still makes sense to still have an eboss_mock mode for future bookkeeping reference. Also, this way we could freeze the 'eBOSS mocks' so that they are still readable even if we make changes to DESI mocks

[moonlovist]

Hi @iprafols @alxogm ,

The issue still exists. I try to run the following command to create the delta fields for a Saclay mock:

picca_deltas.py --in-dir /global/project/projectdirs/desi/mocks/lya_forest/develop/saclay/v4.7/v4.7.10/eboss-0.2/spectra-16/ --drq /global/project/projectdirs/desi/mocks/lya_forest/develop/saclay/v4.7/v4.7.10/eboss-0.2/zcat_desi_drq.fits --dla-vac /global/project/projectdirs/desi/mocks/lya_forest/develop/saclay/v4.7/v4.7.10/eboss-0.2/zcat_desi_drq_DLA_sup_203.fits --out-dir /global/cscratch1/sd/tanting/Result_hcd/Stack_1_10_meanflux_2/v4.7.10/mock_hcd_mask_100_rebins_2/Delta --iter-out-prefix /global/cscratch1/sd/tanting/Result_hcd/Stack_1_10_meanflux_2/v4.7.10/mock_hcd_mask_100_rebins_2/delta_attributes --log /global/cscratch1/sd/tanting/Result_hcd/Stack_1_10_meanflux_2/v4.7.10/mock_hcd_mask_100_rebins_2/input.log --mode desi --nit 20 --rebin 2 --npix-min 75

No matter what mode I choose: desi or desi_mocks, I will get the following error message in the attached document. picca_error message.txt

[alxogm]

Hi @moonlovist,sorry for the delay, just catching up with a lot of things. I'm taking a look.