Additional scripts needed

[KrisThielemans]

• [x] create separate script SIRF_data_preparation/download_Siemens_mMR_NEMA_IQ.py to download/mMR_NEMA_IQ
  • download from zenodo (use zenodo_get)
  • copy mMR_template_span11.hs from sirf_data_path = os.path.join(examples_data_path('PET'), 'mMR')
• [x] create script SIRF_data_preparation/create_initial_images.py to run OSEM and kappa and save to disk(OSEM_image.hv, kappa.hv) takes as arguments
  • a folder-name where all the data is
  • an image-name for image size (for NEMA IQ will be the mumap as in the BSREM notebook)
• [x] NEMA_EQ specific: run script to generate ROIs
• [x] create script SIRF_data_preparation/create_converged_image.py to run BSREM for along time, see #16
  • [x] compute metrics: CIL callback using TensorBoard

Prepared data (sinograms, OSEM_image, kappa, some plots) are at https://petric.tomography.stfc.ac.uk/data/

TODO

• [x] overall penalty weight is fixed to 1/700, might not be ok for future data-sets. Save the value somehow (e.g. simple text file), and read it from the script?, (done in #16 as optionally reading file)
• [x] Clean-up BSREM scripts https://github.com/SyneRBI/PETRIC/tree/BSREM_scripts - the 3 datasets are identical aside from the slice_number for the plot and the num_subsets (should be parameterised?), see #16
  • [x] merge common functionailty into one file
  • [x] use new callbacks & tensorboard logging
  • [ ] drop old-style interface of Algorithm (algo.x, algo.loss)?
• [ ] End-to-end example
  • [ ] example_BSREM.ipynb
  • [x] main.py aka main_BSREM.py (user-configurable) #19
  • [x] petric.py (read-only, we overwrite any user changes)
• [ ] another example but also showing ISTA etc
  • [x] example_ISTA.ipynb, main_ISTA.py
• [x] Update wiki to give instructions on notebook, getting data, create a status page (as we’ll need to give some more updates later). saying that from today -> added here https://github.com/SyneRBI/PETRIC/wiki/Recent-Updates
  • Registration is open
  • 3 example datasets are available (without VOIs), more to come
  • Docker build is edge (no release yet). Manual build is DEVEL_BUILD=ON.
  • Final threshold on metrics TBC
• [x] Insert "Edo's link" at https://github.com/SyneRBI/PETRIC/wiki
• [x] Make this repo public
• [x] Final consistency check
• [x] Announce

[KrisThielemans]

Current status on cdcl machine

download

python download_Siemens_mMR_NEMA_IQ.py 

creates

• .../PETRIC/data/NEMA_IQ.zip
• .../PETRIC/data/NEMA_IQ/ (unzipped)

create acquisition data etc

first edit data_utilities.py to set DATA_PATH (see #15), then

python prepare_mMR_NEMA_IQ_data.py

creates in dir that you set for DATA_PATH, which I set to ~/work/PETRIC (files should be in mMR/NEMA_IQ` subfolder of that)

20170809_NEMA_MUMAP_UCL.hv
20170809_NEMA_MUMAP_UCL.v
20170809_NEMA_UCL.n
additive_term.hs
additive_term.s
intermediate
md5sums.txt
mMR_template_span11.hs
mMR_template_span11.s
mMR_template_span11_small.hs
mMR_template_span11_small.s
multfactors.hs
multfactors.s
norm.n.hdr
prompts_f1g1d0b0.hs
prompts_f1g1d0b0.s
prompts.hs
prompts.s

There's far too many files there. (Edit: fixed in 871346ba36cf324a1e943ba84c74819df4ec8920)

create OSEM

python create_initial_images.py ~/work/PETRIC/data -t 20170809_NEMA_MUMAP_UCL.hv

fails as filename conventions don't match. Fix: multfactors->mult_factors. (fixed in e6dd43a1209140d708917e225d92d4ce161c58fa)

The OSEM_image output was nonsense however (min -2E35, max 0). This is due to https://github.com/UCL/STIR/issues/1280

generate ROIs

find where SIRF-Contribs is from $PYTHONPATH (or later do from python)

cd ~/work/PETRIC/data
python ~/devel/install/sirf/contrib/NEMA/generate_nema_rois.py -i OSEM_image.hv

run BSREM

no script yet.

[KrisThielemans]

Using https://github.com/UCL/STIR/pull/1463, initialisation is fine. However, OSEM gives a ring-artefact at the edge of the FOV currently, but when forcing image-size to xy-size 200

python ~/devel/PETRIC/SIRF_data_preparation/create_initial_images.py ~/devel/PETRIC/data -t 20170809_NEMA_MUMAP_UCL.hv --xy-size 200

it's fine. ROIs are also fine

im=STIR.ImageData('../data/OSEM_image.hv')
S6=STIR.ImageData('../data/S6.hv')
...
plt.imshow(im.as_array()[72,:,:],alpha=((S1+S3+S5+S6).as_array()[72,:,:]+1)/2)

[KrisThielemans]

I've put a preliminary script for BSREM on the BSREM_scripts branch.

Left: 80 updates, middle: 660 updates, right: diff (at colour scale (-max,max)/20)

(im80-im660).abs().max()/im660.max()
np.float32(0.022381468)

[KrisThielemans]

NeuroLF with some corrections. I used exactly same settings, but BSREM subsets 16

Objective function still a bit to go.

This took 1.5s per update on tomography.stfc

[KrisThielemans]

Vision 600 data with same settings, but 5 subsets. 80, 580 and 660 updates. (sorry, diff image failed) about 6s per update

[KrisThielemans]

For the record: data for mMR NEMA IQ was generated as follows:

cd ~/devel/PETRIC/SIRF_data_preparation
python download_Siemens_mMR_NEMA_IQ.py
python prepare_mMR_NEMA_IQ_data.py
DATADIR=~/devel/PETRIC/data/Siemens_mMR_NEMA_IQ/final
python create_initial_images.py $DATADIR -t ../processing/20170809_NEMA_MUMAP_UCL.hv --xy-size 200
pushd $DATADIR
python ~/devel/install/python/sirf/contrib/NEMA/generate_nema_rois.py -i OSEM_image.hv
rm *ahv *nii unregistered*
popd

However, I'll put the output in the shared location.

[KrisThielemans]

Latest mMR results (slightly noisier data). 80, 580 and 660 updates and diff between the last 2 (at -max/20,max/20)

[KrisThielemans]

all done with this!