Part 1 of #98 - Updates to coron1pipeline

[JarronL]

Part 1 of #98.

Added a number of updates to the default coron1pipeline, most importantly the exclusion of dark subtraction for subarrays. This is implemented through a new function run_single_file which sets a number of best-practice defaults, such as skipping darks, IPC, and persistence steps. It also sets some defaults for jump thresholds, saturation flagging parameters, and pseudo-ref pixel rows / columns. run_obs now calls run_single_file.

All of the changes here should not change the outcome of existing notebooks and scripts. The only thing I've done is include better defaults, added more options for simplicity, debugging, and and fixed some known bugs within the pipeline itself.

New pipeline options:

• Can now save intermediate step files via pipe.save_intermediates=True. Similar to the already existing pipe.save_save_calibrated_ramp but for all steps.
• The saturation step now flags RC pixels. Not sure why this isn't included in the default pipeline, but these pixels are persistently hot pixels and should be flagged automatically.
• The pipe.rate_int_outliers utilizes a new cube_outlier_detection function that flags outliers by comparing ramp integrations within an exposure with at least 5 integrations.
• The pipeline can now return the rateint datamodel instead of the default rate datamodel by setting pipe.return_rateints=True. Great for pipeline processing of a single file where you don't actually want to immediately inspect the results without have to save FITS files.

Changes to existing pipeline steps:

• The pseudo refpix correction had some bugs within the indexing logic. I have cleaned it up to make the indexing more consistent and explicit.
• The saturation step now uses a new generic expand_mask function that will grow a binary mask using scipy's binary_dilation function, which is more efficient than array shifting via numpy.

Changes to runobs:

• Calls run_single_file so inherits those defaults described above.
• Added various verbosity options for those who dislike the deluge of log outputs
  • verbose keyword set to True shows everything at INFO level or above from the pipeline. Setting this to False will show various SpaceKLIP log messages, but suppress pipeline message below ERROR.
  • quiet suppresses all messages below ERROR and employs progress bars to keep track of data reduction progress.
  • runobs Skip pipeline processing if data results already exist in output directory rather than re-running unnecessary and overwriting. Can be overriden using the overwrite keyword.

[JarronL]

I realized there were going to be a number of merge conflicts between this PR and 1/f noise correction PR, so I have preemptively merged #102 into here.

Correct 1/f noise striping in stage1:

• Create fnoise_clean.py file that houses code for fitting a 1/f noise model to data and iteratively masks outlier pixels.
• Add cube_fit function, which utilizes polynomial fitting routines from webbpsf_ext to fit linearized unsaturated and unflagged pixel data to quickly generate a kTC bias and slope images.
• Utilize cube_fit to create kTC bias to subtract from linearized ramp data to create clean group-level data
• The slope images from cube_fit are then averaged together and used to produce a high-SNR ramp such that group signal levels match those within a given integration.
• Use 1/f noise filtering algorithm on group level data to model the spatial 1/f striping and subtract it.

1/f cleaning routine adds about 100 msec for a single 320x320 frame. Expect time to increase proportional to frame size. See issue #97 for more information on operations.