Galaxia woes

[michaelJwilson]

mockmaker.readmock('/global/cfs/cdirs/desi/mocks/mws/galaxia/alpha/v0.0.6/healpix/', healpixels=206, nside=8) ...:
INFO:mockmaker.py:1921:readmock: Reading /global/cfs/cdirs/desi/mocks/mws/galaxia/alpha/v0.0.6/healpix/8/2/206/mock_allsky_galaxia_desi-8-206.fits CRITICAL:targets.py:118:encode_targetid: Invalid range when making targetid: OBJID cannot exceed 4194303

---

OSError Traceback (most recent call last)

in 1 for x in healpixels: ----> 2 mockmaker.readmock('/global/cfs/cdirs/desi/mocks/mws/galaxia/alpha/v0.0.6/healpix/', healpixels=206, nside=8) 3 /global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/desitarget/mock/mockmaker.py in readmock(self, mockfile, healpixels, nside, nside_galaxia, target_name, magcut, faintstar_mockfile, faintstar_magcut, seed) 1924 1925 objid = np.arange(nobj) -> 1926 mockid = encode_targetid(objid=objid, brickid=pixnum, mock=1) 1927 1928 allpix = footprint.radec2pix(nside, radec['RA'], radec['DEC']) /global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/desitarget/targets.py in encode_targetid(objid, brickid, release, mock, sky, gaiadr) 117 if 'cannot' in msg: 118 log.critical(msg) --> 119 raise IOError(msg) 120 121 # ADM set up targetid as an array of 64-bit integers. OSError: Invalid range when making targetid: OBJID cannot exceed 4194303

[geordie666]

Indeed, we carefully chose a definition of TARGETID to allow a reasonable maximum for the number of objects in a brick. There cannot be more than ~4 million objects in a "portion" of the sky (a brick or HEALPixel). That's a hard-coded decision at this point, I would think.

Try nside=16? Or whatever gives you fewer than 4.2 million targets in a pixel. Otherwise the mocks will need a different TARGETID definition to the real data (we could rig that, we do have a few spare bits in TARGETID, but it would be messy).

[michaelJwilson]

Makes sense. The Galaxia mock are already packaged at 8, which isn't a decision I can lightly change I guess. I leave it for @moustakas to call (the latter doesn't sound like a route we should go down).

The bug above I originally found trying to create a minimal example that would reproduce this independent error, but I haven't been able to achieve that yet, in the meantime:

Traceback (most recent call last): File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/bin/mpi_select_mock_targets", line 4, in import('pkg_resources').run_script('desitarget==0.42.0', 'mpi_select_mock_targets') File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/conda/lib/python3.8/site-packages/pkg_resources/init.py", line 667, in run_script self.require(requires)[0].run_script(script_name, ns) File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/conda/lib/python3.8/site-packages/pkg_resources/init.py", line 1464, in run_script exec(code, namespace, namespace) File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/EGG-INFO/scripts/mpi_select_mock_targets", line 173, in targets_truth(params, output_dir=args.output_dir, seed=rankseeds[i], File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/desitarget/mock/build.py", line 905, in targets_truth data, MakeMock = read_mock(params['targets'][target_name], log, target_name, File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/desitarget/mock/build.py", line 172, in read_mock data = MakeMock.read(mockfile=mockfile, mockformat=mockformat, File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/desitarget/mock/mockmaker.py", line 3857, in read data = MockReader.readmock(mockfile, target_name=self.objtype, File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/desitarget/mock/mockmaker.py", line 1328, in readmock ra, dec, allpix, pixweight = _get_radec(mockfile, nside, self.pixmap) File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/0.42.0/lib/python3.8/site-packages/desitarget-0.42.0-py3.8.egg/desitarget/mock/mockmaker.py", line 236, in _get_radec allpix = footprint.radec2pix(nside, ra, dec) File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desimodel/0.13.0/lib/python3.8/site-packages/desimodel-0.13.0-py3.8.egg/desimodel/footprint.py", line 96, in radec2pix theta, phi = np.radians(90-dec), np.radians(ra)

[moustakas]

I may not be understanding the error. Although the Galaxia mocks are organized on-disk into nside=8 pixels, select_mock_targets can read arbitrarily sized healpixelized chunks of sky. Specifically, if you simulate nside=16 the code will go and figure out which pixels it needs to read from disk and pull out just the rows it needs.

So you should be able to circumvent / prevent this memory / bit overflow on TARGETID, I think.

[moustakas]

@michaelJwilson I'd like to go ahead and close this ticket given the solution I proposed but feel free to reopen it if it's still on the critical path.