Fix manual MS lesion segmentation from dataset version: `7570a2b796aedff16316645e6a85877486c76752`

[jcohenadad]

Working branch on git-annex: jca/update-groundtruth.

P091 and P167 do not have manual MS lesion segmentation file. They should exist (even if they are empty).

Batch commands

Run in folder basel-mp2rage. Replace "START", "END" with correct numbers.

START=66
END=75

Copy derivatives for review:

for i in $(seq -f "%03g" $START $END); do cp <PATH>/sub-P${i}/anat/sub-P${i}_UNIT1_lesion-manualNeuroPoly.nii.gz derivatives/labels/sub-P${i}/anat/sub-P${i}_UNIT1_lesion-manualNeuroPoly.nii.gz; done

Denoise:

for i in $(seq -f "%03g" $START $END); do sct_maths -i sub-P${i}/anat/sub-P${i}_UNIT1.nii.gz -denoise p=1,b=2 -o sub-P${i}/anat/sub-P${i}_UNIT1_denoised.nii.gz; done

While denoising is running, start running QC segmentations (wait for the first denoising to finish), For easier QCing, we recommend setting up the scene shown below and saving it as qc-sagittal:

Note: This scene will eventually be called by a proper script. See https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A2=ind2301&F=5496681621512503&L=FSL&O=D&P=31427&X=3FB009402978E6EF53&Y=julien.cohenadad%40gmail.com

for i in $(seq -f "%03g" $START $END); do fsleyes -S --scene qc-sagittal sub-P${i}/anat/sub-P${i}_UNIT1_denoised  derivatives/labels/sub-P${i}/anat/sub-P${i}_UNIT1_lesion-manualHaris -cm yellow derivatives/labels/sub-P${i}/anat/sub-P${i}_UNIT1_lesion-manualNeuroPoly -cm red; done

Once manual correction is done, remove the denoised data:

find . -name "*denoised.*" -delete

And push the modified data to git-annex

git add .
git commit -m "Added manual GT from Michelle/Julien: 66->75"
git annex copy --to=origin 
git annex sync --no-content --only-annex
git push

[jcohenadad]

[jcohenadad]

Reviewing Michelle's segmentation.

Batch QCing command (replace 37 with start/end subjects numbers):

for i in {37..37}; do fsleyes -S sub-P0${i}/anat/sub-P0${i}_UNIT1 derivatives/labels/sub-P0${i}/anat/sub-P0${i}_UNIT1_lesion-manualNeuroPoly derivatives/labels/sub-P0${i}/anat/sub-P0${i}_UNIT1_lesion-manualHaris -cm red -a 50; done

[jcohenadad]

lesion missed by Haris/Michelle:

lesion missed by Michelle:

lesion under-segmented by Michelle-- the presentation of a lesion should be more inside the tissue rather than at the periphery only (partial volume with CSF):

[jcohenadad]

Lesion missed by Michelle:

[jcohenadad]

Mislabelling central canal (which appears as hypointense signal) for an MS lesion:

[jcohenadad]

No need to segment above the spinal cord:

[jcohenadad]

Not a lesion:

In general, lesions should be "3D blobs", not 2D blob that only spans a single slice as is the case here. Most lesions should have the shape of a 3D blob, so it is important to work on the 3 views of the viewer to ensure the consistency of the shape.

[mchen1110]

Thank you for the feedback, Julien!

I have been doing the following procedure in regards to lesion segmentation:

1. Use sct_run_batch to process a lesion correction script that opens the image in FSLeyes along with 2 segmentation overlays (lesion-manualHaris and a copy of lesion-manualHaris that is renamed and saved as lesion-manualNeuroPoly).
2. Adjust brightness and contrast to better visualize lesions.
3. Scroll through the views without any overlays to get an idea of what the spinal cord looks like.
4. After identifying areas of hypointensity and using location and presence across multiple slices in all 3 views to confirm the extent of the lesion, use the labeling tool to mark the lesion.
5. Save the lesion segmentation.

[jcohenadad]

to process a lesion correction script

can you pls share this script? or add it in your comment?

[jcohenadad]

As per #46 we realized that running a bit of denoising helps with visual inspection, so here's the SHELL script to run it on the whole dataset:

for i in {33..42}; do sct_maths -i sub-P0${i}/anat/sub-P0${i}_UNIT1.nii.gz -denoise p=1,b=2 -o sub-P0${i}/anat/sub-P0${i}_UNIT1_denoised.nii.gz; done

[mchen1110]

Thank you for the feedback, Julien!

I have been doing the following procedure in regards to lesion segmentation:

1. Use sct_run_batch to process a lesion correction script that opens the image in FSLeyes along with 2 segmentation overlays (lesion-manualHaris and a copy of lesion-manualHaris that is renamed and saved as lesion-manualNeuroPoly).
2. Adjust brightness and contrast to better visualize lesions.
3. Scroll through the views without any overlays to get an idea of what the spinal cord looks like.
4. After identifying areas of hypointensity and using location and presence across multiple slices in all 3 views to confirm the extent of the lesion, use the labeling tool to mark the lesion.
5. Save the lesion segmentation.

The script was created with the help of @sandrinebedard and can be found below. I added in denoising as well based on our discussion. It may need some editing so any feedback is appreciated!

#!/bin/bash
#
# Process data.
#
# Usage:
#   ./correct_lesions.sh <SUBJECT>
#
# Manual segmentations or labels should be located under:
# PATH_DATA/derivatives/labels/SUBJECT/anat/
#
# Authors: Sandrine Bédard, Julien Cohen-Adad

set -x
# Immediately exit if error
set -e -o pipefail

# Exit if user presses CTRL+C (Linux) or CMD+C (OSX)
trap "echo Caught Keyboard Interrupt within script. Exiting now.; exit" INT

# Retrieve input params
SUBJECT=$1

# Save script path
PATH_SCRIPT=$PWD

# get starting time:
start=`date +%s`

# SCRIPT STARTS HERE
# ==============================================================================
# Display useful info for the log, such as SCT version, RAM and CPU cores available
sct_check_dependencies -short

# Go to folder where data will be copied and processed
cd ${PATH_DATA_PROCESSED}
# Copy list of participants in processed data folder
if [[ ! -f "participants.tsv" ]]; then
  rsync -avzh $PATH_DATA/participants.tsv .
fi
# Copy list of participants in resutls folder
if [[ ! -f $PATH_RESULTS/"participants.tsv" ]]; then
  rsync -avzh $PATH_DATA/participants.tsv $PATH_RESULTS/"participants.tsv"
fi
# Copy source images
rsync -avzh $PATH_DATA/$SUBJECT .
# Go to anat folder where all structural data are located
cd ${SUBJECT}/anat/

# MP2RAGE
# ------------------------------------------------------------------------------
file="${SUBJECT}_UNIT1"

# Denoise image
sct_maths -i ${file}.nii.gz -denoise p=1,b=2 -o ${file}_denoised-p1b2.nii.gz

file_denoised="${file}_denoised-p1b2"

FILESEG="${SUBJECT}_UNIT1_lesion-manualHaris"
FILESEGMANUAL="${PATH_DATA}/derivatives/labels/${SUBJECT}/anat/${FILESEG}.nii.gz"
echo
echo "Looking for manual segmentation: $FILESEGMANUAL"
if [[ -e $FILESEGMANUAL ]]; then
  echo "Found! Using manual segmentation."
  rsync -avzh $FILESEGMANUAL ${FILESEG}.nii.gz
  cp ${FILESEG}.nii.gz ${SUBJECT}_UNIT1_lesion-manualNeuroPoly.nii.gz
  fsleyes ${file_denoised}.nii.gz ${FILESEG}.nii.gz ${SUBJECT}_UNIT1_lesion-manualNeuroPoly.nii.gz &

  rsync -avzh ${SUBJECT}_UNIT1_lesion-manualNeuroPoly.nii.gz "${PATH_DATA}/derivatives/labels/${SUBJECT}/anat/${SUBJECT}_UNIT1_lesion-manualNeuroPoly.nii.gz"

fi

# Verify presence of output files and write log file if error
# ------------------------------------------------------------------------------
FILES_TO_CHECK=(

)
pwd
for file in ${FILES_TO_CHECK[@]}; do
  if [[ ! -e $file ]]; then
    echo "${SUBJECT}/anat/${file} does not exist" >> $PATH_LOG/_error_check_output_files.log
  fi
done

# Display useful info for the log
end=`date +%s`
runtime=$((end-start))
echo
echo "~~~"
echo "SCT version: `sct_version`"
echo "Ran on:      `uname -nsr`"
echo "Duration:    $(($runtime / 3600))hrs $((($runtime / 60) % 60))min $(($runtime % 60))sec"
echo "~~~"

[jcohenadad]

The script looks good, but few comments:

• i suggest to run the denoising asynchronously as I did (see https://github.com/ivadomed/model_seg_ms_mp2rage/issues/56#issuecomment-1362105668), that way you don't waste a few minutes at each loop iteration

[mchen1110]

When I tried to run the script, this was the output but the error log was empty:

(env) mchen@LAPTOP-FRUL1V8F:/mnt/c/Users/miche/mp2rage-results$ sct_run_batch -jobs 1 -path-data /mnt/c/Users/miche/basel-mp2rage/ -path-output /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg -script mp2rage_correct_lesions.sh -include sub-P043

--
Spinal Cord Toolbox (git-master-b8d950994081b5a446e2b90c015e3bfdd3ca27ce*)

sct_run_batch -jobs 1 -path-data /mnt/c/Users/miche/basel-mp2rage/ -path-output /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg -script mp2rage_correct_lesions.sh -include sub-P043
--

INFO SYSTEM
-----------
OS: linux (Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
CPU cores: Available: 8 | Threads used by ITK Programs: 1
RAM: Total 7872 MB | Available 7303 MB | Used 296 MB

INPUT ARGUMENTS
---------------
batch_log: sct_run_batch_log.txt
config: null
continue_on_error: 1
email_from: null
email_host: smtp.gmail.com:587
email_to: null
exclude: null
exclude_list: null
ignore_ses: false
include: sub-P043
include_list: null
itk_threads: 1
jobs: 1
path_data: /mnt/c/Users/miche/basel-mp2rage/
path_output: /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg
path_segmanual: .
script: mp2rage_correct_lesions.sh
script_args: ''
subject_prefix: sub-
v: 1
zip: false

SCRIPT
------
git commit: ?!?
git origin: ?!?
Copying script to output folder...
/mnt/c/Users/miche/mp2rage-results/mp2rage_correct_lesions.sh -> /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg/mp2rage_correct_lesions.sh
Setting execute permissions for script file mp2rage_correct_lesions.sh ...

DATA
----
git commit: 7570a2b796aedff16316645e6a85877486c76752*
git origin: git@data.neuro.polymtl.ca:datasets/basel-mp2rage

RUNNING
-------
Processing 1 subjects in parallel. (Worker processes used: 1).
Started at 08h34m44s: sub-P043. See log file /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg/log/mp2rage_correct_lesions_sub-P043.log

Your batch completed but some subjects may have not completed successfully, please consult the logs for:
sub-P043

Started: 08h34m44s | Ended: 08h34m44s | Duration: 00h00m00s

To open the Quality Control (QC) report in a web-browser, open the following file:
/mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg/qc/index.html

[jcohenadad]

first thing to check is: did you download the data from git annex. Can you « ls -la » into the directory of that patient so i can see the size of the file

[mchen1110]

When I tried to run the script, this was the output but the error log was empty:

(env) mchen@LAPTOP-FRUL1V8F:/mnt/c/Users/miche/mp2rage-results$ sct_run_batch -jobs 1 -path-data /mnt/c/Users/miche/basel-mp2rage/ -path-output /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg -script mp2rage_correct_lesions.sh -include sub-P043

--
Spinal Cord Toolbox (git-master-b8d950994081b5a446e2b90c015e3bfdd3ca27ce*)

sct_run_batch -jobs 1 -path-data /mnt/c/Users/miche/basel-mp2rage/ -path-output /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg -script mp2rage_correct_lesions.sh -include sub-P043
--

INFO SYSTEM
-----------
OS: linux (Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid)
CPU cores: Available: 8 | Threads used by ITK Programs: 1
RAM: Total 7872 MB | Available 7303 MB | Used 296 MB

INPUT ARGUMENTS
---------------
batch_log: sct_run_batch_log.txt
config: null
continue_on_error: 1
email_from: null
email_host: smtp.gmail.com:587
email_to: null
exclude: null
exclude_list: null
ignore_ses: false
include: sub-P043
include_list: null
itk_threads: 1
jobs: 1
path_data: /mnt/c/Users/miche/basel-mp2rage/
path_output: /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg
path_segmanual: .
script: mp2rage_correct_lesions.sh
script_args: ''
subject_prefix: sub-
v: 1
zip: false

SCRIPT
------
git commit: ?!?
git origin: ?!?
Copying script to output folder...
/mnt/c/Users/miche/mp2rage-results/mp2rage_correct_lesions.sh -> /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg/mp2rage_correct_lesions.sh
Setting execute permissions for script file mp2rage_correct_lesions.sh ...

DATA
----
git commit: 7570a2b796aedff16316645e6a85877486c76752*
git origin: git@data.neuro.polymtl.ca:datasets/basel-mp2rage

RUNNING
-------
Processing 1 subjects in parallel. (Worker processes used: 1).
Started at 08h34m44s: sub-P043. See log file /mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg/log/mp2rage_correct_lesions_sub-P043.log

Your batch completed but some subjects may have not completed successfully, please consult the logs for:
sub-P043

Started: 08h34m44s | Ended: 08h34m44s | Duration: 00h00m00s

To open the Quality Control (QC) report in a web-browser, open the following file:
/mnt/c/Users/miche/mp2rage-results/2022-12-20-manual-seg/qc/index.html

After some investigations with @valosekj, we received an error message: -bash: ./mp2rage_correct_lesions.sh: /bin/bash^M: bad interpreter: No such file or directory. We ran the following command to remove the ^M carriage return character: sed -i -e 's/\r$//' mp2rage_correct_lesions.sh and the script ran without any issues

[jcohenadad]

false positive:

[jcohenadad]

False negative

[jcohenadad]

false positive (canal spinal).

[jcohenadad]

Good catch! (although slightly oversegmented)

[jcohenadad]

False positive:

[jcohenadad]

Lesions too big. See animation below toggling between: Image, Michelle, Julien:

[jcohenadad]

These are two separate lesions-- they should not be attached:

[jcohenadad]

@mchen1110 Here is a screen recording that shows my approach to manual label creation. Hopefully this will be helpful.

[jcohenadad]

@mchen1110 Here are my modified segmentations from your segmentations:

 M derivatives/labels/sub-P124/anat/sub-P124_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P130/anat/sub-P130_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P132/anat/sub-P132_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P134/anat/sub-P134_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P138/anat/sub-P138_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P139/anat/sub-P139_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P141/anat/sub-P141_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P142/anat/sub-P142_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P146/anat/sub-P146_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P153/anat/sub-P153_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P165/anat/sub-P165_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P169/anat/sub-P169_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P173/anat/sub-P173_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P174/anat/sub-P174_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P176/anat/sub-P176_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P187/anat/sub-P187_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P190/anat/sub-P190_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P243/anat/sub-P243_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P249/anat/sub-P249_UNIT1_lesion-manualNeuroPoly.nii.gz
 M derivatives/labels/sub-P250/anat/sub-P250_UNIT1_lesion-manualNeuroPoly.nii.gz

Overall it looked excellent!

[jcohenadad]

New ground truths uploaded on master:

Terminal output

```console ef09a994b1bd8dae70fea7d811c89dfae7f52025 (HEAD -> master, origin/master, origin/HEAD) Correction Julien: 132->250 757f0494ba9793f9828d79f7d1fdb41c149f229c Correction Julien: 122->130 5666dd67c7149bf0a7e4b2d5a7f0550c50252b5b Added manual GT from Michelle: 119->250 20da1a9e16a3f001c8d41b17ce5655ab160688bc Added manual GT from Michelle/Julien: 109->118 0b628c2c2835dfa2f21d930666ec1320cec5c10c Added manual GT from Michelle/Julien: 98->108 49f8112785318eab72acb3974bbb49fac207434d Added manual GT from Michelle/Julien: 86->97 437149565b2f10a8220db5010e37ee0c3f2c70fb Added manual GT from Michelle/Julien: 76->85 0565b2d437d69e97876e41ca6677b6693bcf9a74 Added manual GT from Michelle/Julien: 66->75 04aefadfcb637da55bcf447fac5a28f678f2f2a0 Added manual GT from Michelle/Julien: 54->65 e81501df6197b40e812007d308e64f7259af4f49 Added manual ground truth from Michelle/Julien: 43->53 44f15be8c8b3cf6a4045b3a7e86457f1eb41eba0 Added manual segmentation from NeuroPoly ```