Quick summary of raw output

bin/qa-fiberassign is a start towards this. It first checks for outright errors (e.g. same target assigned to multiple fibers on the same tile) and then proceeds to generate some stats. Several items to do:

check usability on full 16k tiles (I've only tested it with a mini 22 tile set)
more metrics, e.g. fraction of targets types assigned on dark/gray/bright tiles
more error checks, e.g. that there are enough sky and standards assigned
don't count bright tile coverage of dark time targets and vice-a-versa
output json or yaml data file with summary statistics
plots, HTML wrappers
refactor script into fiberassign.qa python library that could be called, e.g. from quicksurvey

Example current outputs on a 22 tile sample with lots of edge effects (i.e. don't panic about low efficiencies):

[cori12 end2end] qa-fiberassign fiberassign/tile*.fits --targets targets/targets.fits 
fiberassign/tile_01148.fits - OK
fiberassign/tile_01149.fits - OK
fiberassign/tile_01165.fits - OK
fiberassign/tile_06911.fits - ERROR
  2 unassigned fibers
fiberassign/tile_06927.fits - ERROR
  1 unassigned fibers
...
16/22 tiles had errors
----------
60.2% targets (207595/344701) covered by 22 tiles
  Dropping 137106 targets not covered by any tile:fiber
Fiber assignment efficiency
  ELG       41.2%  (35766/86859)
  LRG       81.1%  (9049/11154)
  QSO       96.0%  (5769/6012)
  MWS_ANY   15.9%  (7509/47157)
  BGS_ANY   51.3%  (30598/59589)

---------------------------------------------------------------
WORK IN PROGRESS
The stats below mix dark/bright/gray without considering that a
dark target can't be observed on a bright tile
---------------------------------------------------------------

Percent of targets covered N times
            1     2     3     4     5     6     7     8     9
          23.2  23.1  17.2  13.3   9.8   5.6   3.5   2.7   1.2

Percent of targets covered at least N times
            1     2     3     4     5     6     7     8     9
         100.0  76.8  53.7  36.5  23.2  13.5   7.9   4.4   1.8

Assignment probability if covered at least N times
            1     2     3     4     5     6     7     8     9
ELG        3.9  11.1  18.6  28.6  37.5  40.5  41.0  41.2  41.2
LRG       16.3  38.1  59.6  76.0  80.4  81.0  81.1  81.1  81.1
QSO       22.5  48.1  70.5  88.2  94.3  95.6  95.9  96.0  96.0
MWS_ANY    0.5   1.3   2.2   3.4   5.3   7.9  10.7  13.6  15.1
BGS_ANY    9.7  23.3  35.0  38.1  40.6  43.6  46.5  49.3  50.7

i.e. this is a work in progress; please feel free to expand it with the metrics that you would like to see.

Note: the fiber assignment C++ code has some similar metrics built in, but it is still useful to have an independent cross check that the as-written files are correct.

apcooper / bright_analysis

Quick summary of raw output #2