desihub / desitarget

DESI Targeting
BSD 3-Clause "New" or "Revised" License
18 stars 23 forks source link

Speed-ups and bug-fixes for I/O routines #753

Closed geordie666 closed 3 years ago

geordie666 commented 3 years ago

This PR fixes a bug when reading MTL ledgers. It also implements some speed-ups in the functions that read targets, which were suggested by @araichoor. To summarize:

geordie666 commented 3 years ago

@araichoor: I think you can switch your bespoke code in fba_launch_io.py over to just two lines of code using the following schema:

  1. Read the targets using targets=io.read_targets_in_tiles(hpdirname, tiles=tiles, quick=True) or by directly using targets=io.read_targets_in_tiles_quick(hpdirname, tiles=tiles)
  2. Match the targets and the mtl using the quick version of mtl.inflate_ledger(), which is mtl_with_target_columns = mtl.match_ledger_to_targets(mtl, targets).

You might also want to just try mtl_with_target_columns = mtl.inflate_ledger(mtl, targdir, quick=True) directly, as there doesn't appear to be a huge speed-up from switching to using io.read_targets_in_tiles_quick().

It would be great if you could try this set-up for some pass 0 tiles and check that results are identical and (hopefully) faster than the standard desitarget.io routines.

coveralls commented 3 years ago

Coverage Status

Coverage decreased (-0.2%) to 58.516% when pulling 752dde7f870735906738ed642cbe2361ca1ef9d5 on ADM-speed-IO into 263d22ea11f9df428bf6e2e5208b6681c9719312 on master.