search

desihub / desisurveyops

Scripts and code for DESI Survey Operations
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

EXPID 261455, 261456 marked bad on 20241105 due to mismatch in fiberassign tile and guider tile #246

Open abhi0395 opened 2 weeks ago

abhi0395 commented 2 weeks ago

tilenight job for tileid 10535 (expid: 261454 - 261456) from 20241105 is failing citing following error:

ERROR:util.py:163:runcmd: Traceback (most recent call last):
ERROR:util.py:163:runcmd: File "/global/common/software/desi/perlmutter/desiconda/20240425-2.2.0/code/desispec/main/py/desispec/util.py", line 146, in runcmd
    result = cmd(*args)
ERROR:util.py:163:runcmd: File "/global/common/software/desi/perlmutter/desiconda/20240425-2.2.0/code/desispec/main/py/desispec/scripts/assemble_fibermap.py", line 53, in main
    fibermap = assemble_fibermap(args.night, args.expid, badamps=args.badamps,
ERROR:util.py:163:runcmd: File "/global/common/software/desi/perlmutter/desiconda/20240425-2.2.0/code/desispec/main/py/desispec/io/fibermap.py", line 1050, in assemble_fibermap
    raise RuntimeError('fiberassign tile {} != guider tile {}'.format(
ERROR:util.py:163:runcmd: RuntimeError: fiberassign tile 10535 != guider tile 3842

log file: https://data.desi.lbl.gov/desi/spectro/redux/daily/run/scripts/night/20241105/tilenight-20241105-10535-32610546.log

I checked the HDU for the guider file for expid 261455 and 261456:

fitsheader -e 0 -k TILEID /global/cfs/cdirs/desi/spectro/data/20241105/00261456/guide-00261456.fits.fz
# HDU 0 in /global/cfs/cdirs/desi/spectro/data/20241105/00261456/guide-00261456.fits.fz:
TILEID  =                 3842 / DESI Tile ID   

fitsheader -e 0 -k TILEID /global/cfs/cdirs/desi/spectro/data/20241105/00261455/guide-00261455.fits.fz
# HDU 0 in /global/cfs/cdirs/desi/spectro/data/20241105/00261455/guide-00261455.fits.fz:
TILEID  =                 3842 / DESI Tile ID

@sbailey suggested to mark these two exposures as bad and reprocess the tile for now.

araichoor commented 2 weeks ago

in case it helps, tileid=3842 was the tile observed just before (expids 261452, 261453).

abhi0395 commented 2 weeks ago

Yes, I too saw that, perhaps something happened while writing these files.

araichoor commented 2 weeks ago

dumping more info, for the record: from the nightlog, there s been a guide 8 error between expid 261454 and 261455, maybe related to the issue?

22:54 [05:54] Exp. 261454- Good - Dark program, tile ID 10535. Looks good. Survey speed is between 88% and 105%, transparency varies from 92% to 100%, sky level 2.4, seeing 1.21 arcsec. 1st correction move RMS is 3.2 um, final move RMS is 4.49 um. (SO)
Tile 10535, Exptime: 1258.97, Airmass: 1.71, Sequence: DESI, Flavor: science, Program: DARK

23:05 [06:05] guide 8 error; Action: none needed (LO)

23:17 [06:17] Exp. 261455- Good - Dark program, split exposure of the tile ID 10535. Looks good. Survey speed is between 61.9% and 77.4%, transparency 99%, sky level 2.53, seeing 1.21 arcsec. (SO)
Tile 10535, Exptime: 1120.93, Airmass: 1.75, Sequence: _Split, Flavor: science, Program: DARK