`sct_straighten_spinalcord` cannot seem to straighten a spinal cord that is too short?

[NadiaBlostein]

Branch

nb/issues_59_and_41

Context

sct_straighten_spinalcord seems to fail at straightening spinal cord data that is defined between 14 or less intervertebral discs

sct_straighten_spinalcord seems to fail at straightening spinal cord data that is defined between 14 or less intervertebral discs

In preprocess_normalize.py, the main method calls straighten_all_subjects() on line 683

straighten_all_subjects() then callssct_straighten_spinalcord in preprocess_normalize.py, lines 482-491: This part of the code seems to fail whenever the distance between first_disc and last_disc is less than 15 levels. For instance, if you set up the configuration.json values such that first_disc is 1 and last_disc is 15 or less, the following error occurs:

Traceback (most recent call last):
  File "/home/GRAMES.POLYMTL.CA/p117938/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_straighten_spinalcord.py", line 278, in <module>
    main(sys.argv[1:])
  File "/home/GRAMES.POLYMTL.CA/p117938/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_straighten_spinalcord.py", line 261, in main
    fname_straight = sc_straight.straighten()
  File "/home/GRAMES.POLYMTL.CA/p117938/spinalcordtoolbox/spinalcordtoolbox/straightening.py", line 387, in straighten
    idx_closest = centerline_straight.get_closest_to_absolute_position(disc_label, relative_position,
  File "/home/GRAMES.POLYMTL.CA/p117938/spinalcordtoolbox/spinalcordtoolbox/types.py", line 479, in get_closest_to_absolute_position
    relative_position_from_reference_backup = backup_centerline.dist_points[backup_index] - position_reference_backup
IndexError: list index out of range

Reproducing this problem

• git clone git@github.com:neuropoly/template.git -b nb/issues_59_and_41
• Follow steps 1.1 to 1.6 in README.md
• Make sure to set first_disc to 1 and last_disc to 15 or less in configuration.json
• Run the following: python preprocessing_normalize.py configuration.json

[rohanbanerjee]

Similar issue as #22

[NadiaBlostein]

@rohanbanerjee is getting a similar error as well (associated with discs in the range 23-26). Moving forward:

• Update to SCT 6.0
• Investigate further and open issue on spinalcordtoolbox/spinalcordtoox

[NadiaBlostein]

A similar problem occurs in a separate dataset. In the SCT tutorial, the multisubject data has images that go down to disc 11. Setting the last_disc in configuration.json to 8 or less creates the same problem.

Reproducing this problem with tutorial dataset

1. Download and unzip data_batch-processing-of-subjects.zip from SCT tutorial

2. Install SCT & activate Miniconda environment (Python 3.9.17)

   git clone https://github.com/spinalcordtoolbox/spinalcordtoolbox
   cd spinalcordtoolbox
   git checkout a11e9f1f5675fe6f25f3aceda4ceb5d13ad07c59
   ./install_sct
   cd ..
   conda activate spinalcordtoolbox/python/envs/venv_sct/

3. Current directory (ls):

   • multi_subject
   • spinalcordtoolbox

4. Label SC and discs according to SCT tutorial

   cd multi_subject
   sct_run_batch -script process_data.sh -config config.yml

5. BIDSify the directory

   for sub in data/sub*; do echo 
   rm -R data/
   mkdir derivatives
   mkdir derivatives/labels
   for sub in sub*; do mkdir derivatives/labels/$sub; done
   for sub in sub*; do mkdir derivatives/labels/$sub/anat; done
   for sub in sub*; do for file in output/data_processed/$sub/anat/*_seg.nii.gz; do cp $file derivatives/labels/$sub/anat/$(basename $file _seg.nii.gz)_label-SC_mask.nii.gz; done; done
   for sub in sub*; do for file in output/data_processed/$sub/anat/*_seg_labeled_discs.nii.gz; do cp $file derivatives/labels/$sub/anat/$(basename $file _seg_labeled_discs.nii.gz)_labels-disc.nii.gz; done; done

6. Follow steps 1.1 to 1.6 in README.md

7. Make sure to set first_disc to 1 and last_disc to 5 or less in configuration.json

8. Run the following: python preprocessing_normalize.py configuration.json

[NadiaBlostein]

testing_script.txt

Example

1. Download and unzip data_batch-processing-of-subjects.zip from SCT tutorial

2. Install SCT & activate Miniconda environment (Python 3.9.17)

   git clone https://github.com/spinalcordtoolbox/spinalcordtoolbox
   cd spinalcordtoolbox
   git checkout a11e9f1f5675fe6f25f3aceda4ceb5d13ad07c59
   ./install_sct
   cd ..
   conda activate spinalcordtoolbox/python/envs/venv_sct/

3. Current directory (ls):

   • multi_subject
   • spinalcordtoolbox

4. Label SC and discs according to SCT tutorial

   cd multi_subject
   sct_run_batch -script process_data.sh -config config.yml

5. From the multi_subject, run testing_script.py with different last_disc values to see how the output changes:

   mv testing_script.txt testing_script.py
   python testing_script.py 11 # should give you the correct outputs
   python testing_script.py 7 # throws error

[joshuacwnewton]

Error investigation

The error occurs in the get_closest_to_absolute_position method, when backup_centerline.dist_points[backup_index] is called:

  File "/home/GRAMES.POLYMTL.CA/p117938/spinalcordtoolbox/spinalcordtoolbox/types.py", line 479, in get_closest_to_absolute_position
    relative_position_from_reference_backup = backup_centerline.dist_points[backup_index] - position_reference_backup
IndexError: list index out of range

• backup_index comes from iterating through centerline.number_of_points.
• backup_centerline is just centerline_straight, which in this case comes from the centerline reference file.
• In this case, centerline has more points than centerline_straight (due to cropping from last_disc):
• So backup_index becomes too large, and thus we get an IndexError.
• Files from @NadiaBlostein's testing script:
  • centerline: 'output/data_processed/sub-01/anat/sub-01_T2w_seg.nii.gz'
  • centerline_straight: 'template-last_disc_7/template_label-centerline.nii.gz'

Potential solution

It looks like the outer loop already has a condition for when the return value from get_closest_to_absolute_position is equal to None. So, adding a simple check and returning None is the first thing that came to mind:

FIX

```diff def get_closest_to_absolute_position(self, vertebral_level, relative_position, backup_index, backup_centerline): if vertebral_level == 0: # above the C1 vertebral level, the method used is length label = self.first_label else: vertebral_number = self.labels_regions[vertebral_level] if self.potential_list_labels.index(vertebral_number) < self.list_labels.index(self.first_label): label = self.first_label elif self.potential_list_labels.index(vertebral_number) >= self.list_labels.index(self.last_label): label = self.last_label else: return self.get_closest_to_relative_position(vertebral_level=vertebral_level, relative_position=relative_position) + if backup_index >= backup_centerline.number_of_points: + return None position_reference_backup = backup_centerline.dist_points[backup_centerline.index_disc[backup_centerline.regions_labels[label]]] position_reference_self = self.dist_points[self.index_disc[self.regions_labels[label]]] relative_position_from_reference_backup = backup_centerline.dist_points[backup_index] - position_reference_backup return np.argmin(np.abs(np.array(self.dist_points) - position_reference_self - relative_position_from_reference_backup)) ```

If we add this, the error is no longer thrown. But, does the rest of the straightening go okay? ANS: Surprisingly, yes! This small fix is all that was needed:

SCREENSHOT

 Straightened image (last_disc=7) overlaid on top of straightened image (last_disc=11)

Please check out the SCT branch jn/60-fix-indexerror-during-straightening then re-run any processing to test out the fix. :)

[rohanbanerjee]

Thank you @joshuacwnewton for looking into this! I tested the above branch for my data -- which is the dog spinal cord data. This works well now and I can give any disc value and it runs without any error.

However, I wanted to bring one thing to notice here (perhaps this should be a separate issue, please let me know if I should open a separate issue for this). Even though this works well, I had a small observation that I would like to point out here. Let us consider that I am just using one subject right now and my subject has 26 disc levels. I am using the straighten_spinal_cord to straighten the spinal cord and giving the centerline (instead of the segmentation) and the disc levels as the input for the same. Even though the script produces the straightened spinal cord, it leaves out the last few disc levels. The image below shows this:

This straightened image only has 23 levels but it's expected to have 26. Any thoughts on this?

[joshuacwnewton]

This straightened image only has 23 levels but it's expected to have 26. Any thoughts on this?

Could you please share the command you used, as well as the data files? I'll happily take a look at this issue. :)

[NadiaBlostein]

@joshuacwnewton Just confirming that what you did fixed the error! Amazing, thank you very much!

[NadiaBlostein]

Updated README in commit 14173267ba0a57a131dadcd44b000c05686e1561 (branch nb/issues_59_and_41) to version of SCT that uses @joshuacwnewton's fix. Issue #63 opened to remember to update README once jn/60-fix-indexerror-during-straightening becomes part of stable version of SCT.

@rohanbanerjee will open a new issue related to what he is describing above