Various improvements

[PaulBautin]

This PR does the following:

• correct issue #26 by indicating user via README to fetch dataset from spine-generic https://github.com/spine-generic/data-multi-subject/archive/r20200801.zip
• correct issue for csa_rescale_stat.py (#35, #36, #37)
• freeze config files and README for spine-generic dataset processing

List of TODO:

• [x] use the sct_run_batch variables PATH_RESULTS and PATH_DATA_PROCESSED in process_data.sh

• [x] make csa_rescale_stat.py part of setup.py

• [x] use only one vertebral labeling (rescale = 1) per subject in process_data.sh

• [x] Verify transfo_values.csv can be accessed when running process_data.sh in a new directory (ex: segmentation correction)

• [x] Use spine-generic manual correction #40

FIX #35, FIX #36, FIX #37, FIX #26, FIX #32, FIX #29, FIX #40

[PaulBautin]

@jcohenadad, i would need help to implement the manual correction. How do i process labelling using the manual correction that contains only 2 labels?

[jcohenadad]

How do i process labelling using the manual correction that contains only 2 labels?

ok, so currently, spine-generic has manual manual labels with only 2 labels. You could e.g. use label 3, which corresponds to 2-3 disc (good for your study).

however, not ALL subjects have this label-- the ones where vertebral labeling worked don't have those labels. In that case you want to use the warped vertebral levels as a reference level for your study.

this is a fairly complex question (there are multiple solutions for it, but we need to chose carefully), which i think should deserve its own specific issue.

[jcohenadad]

@PaulBautin i don't understand the purpose of this PR. Could you explicit the title a bit more "fixes for first release" is not specific.

also, i see that this PR includes other PRs:

• merge of PR #28 "fetch dataset"
• merge of PR #30 "large dataset computing" and correction

This is now how GH workflow works. I think I already mentioned it in a previous PR that you should not include a PR in another PR. All PR should be based of the master branch. That's it. No dependencies with other branches please.

So, the proper workflow is:

• you have issue A
• you fix issue A in PR a
• you merge PR a into master
• you update your local branch to master (always do that, please please please please!!!). Why? --> because i don't want to spend time reviewing code that i've already reviewed.
• you have issue B
• you fix issue B in PR b
• you merge PR b into master …

please always follow this scheme.

this means: close this PR (because it does not follow this scheme) and create a specific PR without inclusions from former PRs.

[PaulBautin]

this is a fairly complex question (there are multiple solutions for it, but we need to chose carefully), which i think should deserve its own specific issue.

I have opened an issue on this subject #32

Could you explicit the title a bit more "fixes for first release" is not specific.

My intention with this PR was to gather final fixes and release a version containing config to process spine-generic dataset.

this means: close this PR (because it does not follow this scheme) and create a specific PR without inclusions from former PRs

These former PRs were never merged to master (i had the intention of closing without merging #30 because of the interdependencies). Should i close this PR and dispatch its elements for clarity (I can still use #30 )?