search

ivadomed / canproco

Code for preprocessing the CanProCo brain and spinal cord dataset
MIT License
4 stars 1 forks source link

Create protocol for manual lesion segmentation #73

Open jcohenadad opened 7 months ago

jcohenadad commented 7 months ago

Some relevant discussions:

We should use https://github.com/spinalcordtoolbox/manual-correction

plbenveniste commented 7 months ago

@cspino and I are working on manual correction of MS lesion in the spinal cord predicted by the model (refer to ACTRIMS abstract). So far, the proposed method is the following :

However, we identified certains things to do beforehand:

@jcohenadad Could you give me feedback on the json file data for the automatically segmented lesions

valosekj commented 7 months ago

Great summary! 👍🏻

For SCI derivatives (segmented using the SCISeg model and then manually corrected), I use the following jSON sidecar: 

{
    "GeneratedBy": [
        {
            "Name": "SCISeg nnUNet3D",
            "Version": "Dataset183_tSCI3SitesALPhase2Seed710",
            "Date": "2024-01-16"
        },
        {
            "Author": "Jan Valosek",
            "Note": "Manually corrected",
            "Date": "2024-01-18 10:42:24"
        }
    ]
}

Or, if a mask is created completely manually, I use:

{
    "GeneratedBy": [
        {
            "Author": "Jan Valosek",
            "Date": "2024-01-18 10:42:24"
        }
    ]
}

Unfortunately, manual-correction.py supports only the second JSON option. I usually write a simple loop to add info about a model to the JSON sidecar. UPDATE - an issue opened: https://github.com/spinalcordtoolbox/manual-correction/issues/75

Also see our intranet convention here.

valosekj commented 7 months ago

Modify manual_correction script to change Fsleyes parameters : such as the intensity parameters depending on the contrast

This is already supported; see the -fsleyes-dr flag here.

plbenveniste commented 7 months ago

In our case, what do you think of ? 

{
    "GeneratedBy": [
        {
            "Name": "nnunet_region-based_(SC+lesion)_training_on_STIR_and_inverted_PSIR_data nnUNet2D",
            "Version": "Dataset222_RegionBasedLesionSeg",
            "Date": "2024-01-16"
        },
        {
            "Author": "Pierre-Louis Benveniste",
            "Note": "Manually corrected",
            "Date": "2024-01-18 10:42:24"
        }
    ]
}
jcohenadad commented 7 months ago

"Name": "nnunetregion-based(SC+lesion)_training_on_STIR_and_inverted_PSIR_data nnUNet2D" --> ideally it should point to the actual model (eg: URL to the model hosted on the assets of a published release on github). Or maybe the version should point to the versioned model... i don't know what's best. But in any case we need a link, otherwise in 5 years another student might give the same name to the model, and then we have a conflict

"Note": "Manually corrected", --> i don't think this field is necessary

plbenveniste commented 7 months ago

Release done ! Format of json files will be : 

{
    "GeneratedBy": [
        {
            "Name": "2D nnUNet model stored at https://github.com/ivadomed/canproco/releases/tag/r20240125",
            "Version": "2D region-based nnUNet fold 2",
            "Date": "2024-01-16"
        },
        {
            "Author": "Pierre-Louis Benveniste",
            "Date": "2024-01-18 10:42:24"
        }
    ]
}
jcohenadad commented 7 months ago

Great! I have a comment about naming the model https://github.com/ivadomed/canproco/issues/74

Also, putting the version URL under "Name" defeats the purpose of the field "Version".

How about instead putting the name of the model (ie: the packaged ZIP file, like model_ms_seg_sc-lesion_regionBased.zip) under "Name", and the URL to the release under "Version"?

valosekj commented 7 months ago

"Note": "Manually corrected", --> i don't think this field is necessary

I used this because, in some cases, the automatic seg was okay and needed no corrections. To distinguish this, I used:

    "Note": "Visually verified",

or, if manual correction was required:

    "Note": "Manually corrected",
jcohenadad commented 7 months ago

 "Note": "Visually verified",

I agree, this is a good idea. We should probably add this information in our intranet so as to standardize our terminology @NathanMolinier

BTW: Does your manual_correction script automatically identifies if there was a modification in the segmentation, and then applies either "Visually verified" or "Manually corrected"?

valosekj commented 7 months ago

BTW: Does your manual_correction script automatically identifies if there was a modification in the segmentation, and then applies either "Visually verified" or "Manually corrected"?

Yes! The script tracks the modification times of a file to distinguish if the file was just opened or actually modified.

plbenveniste commented 7 months ago

The json files have the following format : 

{
        "GeneratedBy": [
            {
                "Name": "2D nnUNet model model_ms_seg_sc-lesion_regionBased.zip",
                "Version": "https://github.com/ivadomed/canproco/releases/tag/r20240125",
                "Date": "2024-01-26"
            }
        ]
    }
NathanMolinier commented 7 months ago

"Note": "Visually verified",

I agree, this is a good idea. We should probably add this information in our intranet so as to standardize our terminology @NathanMolinier

BTW: Does your manual_correction script automatically identifies if there was a modification in the segmentation, and then applies either "Visually verified" or "Manually corrected"?

Regarding this point, I discussed with @valosekj and I think that we should stick to our standards and use the field "Name" instead of "Note". For more information about this please check this comment in manual correction.

Manual correction script needs to be updated for sure regarding json sidecars. Also few JSON sidecar examples are presented in our new convention.

NathanMolinier commented 7 months ago

The field "SpatialReference": "orig" should also be added @plbenveniste

plbenveniste commented 7 months ago

Great idea @NathanMolinier ! However, because of consistency, I suggest adding this field to the entire dataset and not just to these specific segmentations. An issue was created #75

NathanMolinier commented 7 months ago

Yes indeed ! All the json sidecars should have this "SpatialReference" 👍

valosekj commented 7 months ago

Regarding the "SpatialReference", you could test manual-correction PR #76, which now adds "SpatialReference" automatically.