option to ignore certain flats/darks

[dkazanc]

we need to add an option to ignore certain flats and/or darks in the loader as sometimes some of them are not usable.

Ideally we would like to combine a singular selection and a batch selection, e.g. in the list: ignore_flats: [5, 8:15, 19] ignore_darks: [0:20]

Also by default it can be: ignore_flats: False ignore_darks: False

but when ignore_flats: True ignore_darks: True ALL flats and darks must be ignored....

Daniil wrote:

you can test it on Diad's data that they just collected, e.g. sample 23565 you will see that the normalisation result will have no Infs if you exclude some darks (which are not darks)

- httomo.data.hdf.loaders:
    standard_tomo:
      name: tomo
      data_path: /entry/imaging/data
      image_key_path: /entry/instrument/imaging/image_key
      rotation_angles:
        data_path: /entry/imaging_sum/gts_cs_theta
      dimension: 1
      preview:
        - 
        - start: 1000
          stop: 1005
        - start: 700
          stop: 2000
      pad: 0
- httomolibgpu.prep.normalize:
    normalize:
      data_in: tomo
      data_out: tomo
      cutoff: 10.0
      minus_log: true
      nonnegativity: false 
- httomolibgpu.recon.algorithm:
    FBP:
      data_in: tomo
      data_out: tomo
      center: 700
      objsize: null      
- httomolib.misc.images:
    save_to_images:
      data_in: tomo
      subfolder_name: images
      axis: 0
      file_format: tif
      bits: 8
      perc_range_min: 0.0
      perc_range_max: 100.0
      jpeg_quality: 95

[dkazanc]

in addition for that particular sample 23565 try to exclude darks like this (this assumes that you have 40 darks in total, 20 in the beginning of the scan and 20 at the end)

ignore_flats: False
ignore_darks: [0:20]

This should pick up only the last series of 20 darks which are OK.