DiamondLightSource / httomo

High-throughput tomography pipeline
https://diamondlightsource.github.io/httomo/
Other
5 stars 4 forks source link

Add parameter sweep functionality #380

Closed yousefmoazzam closed 2 months ago

yousefmoazzam commented 3 months ago

Attempt to fix #362

Main changes:

Things left to do:

Acceptance criteria checklist

yousefmoazzam commented 2 months ago

For an example of how the logging looks so far (I've only added the basics), with the following pipeline (sweeps over 10 center values in the recon):

- method: standard_tomo
  module_path: httomo.data.hdf.loaders
  parameters:
    name: tomo
    data_path: entry1/tomo_entry/data/data
    image_key_path: entry1/tomo_entry/instrument/detector/image_key
    rotation_angles:
      data_path: /entry1/tomo_entry/data/rotation_angle
    dimension: 1
    pad: 0
    preview:
      detector_y:
        start: 50
        stop: 57
- method: normalize
  module_path: httomolibgpu.prep.normalize
  parameters:
    cutoff: 10.0
    minus_log: true
    nonnegativity: false
    remove_nans: false
- method: paganin_filter_tomopy
  module_path: httomolibgpu.prep.phase
  parameters:
    pixel_size: 0.0001
    dist: 50.0
    energy: 53.0
    alpha: 0.001
- method: remove_all_stripe
  module_path: httomolibgpu.prep.stripe
  parameters:
    snr: 3.0
    la_size: 61
    sm_size: 21
    dim: 1
- method: FBP
  module_path: httomolibgpu.recon.algorithm
  save_result: False
  parameters:
    center: !SweepRange
      start: 10
      stop: 20
      step: 1
    filter_freq_cutoff: 0.6
    recon_size: null
    recon_mask_radius: null
- method: save_to_images
  module_path: httomolib.misc.images
  parameters:
    subfolder_name: images
    axis: 1
    file_format: tif
    bits: 8
    perc_range_min: 0.0
    perc_range_max: 100.0
    jpeg_quality: 95
    offset: 0
    asynchronous: true

the following terminal output is produced:

See the full log file at: /dls/tmp/twi18192/sweep-tests/08-07-2024_16_35_08_output/user.log
Loading data with shape (1801, 7, 2560)
Running data_reducer (httomolib)
Running normalize (httomolibgpu)
Running paganin_filter_tomopy (httomolibgpu)
Running remove_all_stripe (httomolibgpu)
Running FBP (httomolibgpu)
    Parameter sweep over 10 values of parameter: center
      0%|          | 0/10 [00:00<?, ?value/s, center=10]
     10%|#         | 1/10 [00:02<00:21,  2.36s/value, center=11]
     20%|##        | 2/10 [00:03<00:12,  1.51s/value, center=12]
     30%|###       | 3/10 [00:04<00:08,  1.25s/value, center=13]
     40%|####      | 4/10 [00:05<00:06,  1.12s/value, center=14]
     50%|#####     | 5/10 [00:06<00:05,  1.04s/value, center=15]
     60%|######    | 6/10 [00:06<00:03,  1.00value/s, center=16]
     70%|#######   | 7/10 [00:07<00:02,  1.03value/s, center=17]
     80%|########  | 8/10 [00:08<00:01,  1.05value/s, center=18]
     90%|######### | 9/10 [00:09<00:00,  1.06value/s, center=19]
    Finished parameter sweep
Running save_to_images (httomolib)
Pipeline finished. Took 32.329s

Feel free to make any suggestions on tweaks/additions for improvement :slightly_smiling_face: