search

CompImg / LST-AI

LST-AI - Deep Learning Ensemble for Accurate MS Lesion Segmentation
https://doi.org/10.1016/j.nicl.2024.103611
MIT License
18 stars 4 forks source link

LST-LGA vs LST-AI: Comparisons #14

Closed pradhanhitesh closed 1 month ago

pradhanhitesh commented 2 months ago

Hello Developers! Thank you for the open-source toolbox. It is amazing to see the potential of LST-AI in segmenting the WMH volumes and further classifying the lesions based on location.

I work as a Research Assistant for the SANSCOG study and have been working on WMH segmentation for quite a long. We had earlier used LST-LGA to perform WMH segmentation and found surprising results when LST reported, for approximately 29% (~428) of the subjects (N=1600), to have "zero" lesion volume. We cross-verified the T2F scans and compared the results with Fazekas Scores, graded by the Radiologist. There was agreement in most of the cases. Recently, we again performed WMH segmentation with LST-AI and found that subjects with "zero" lesion volume (as reported by LST-LGA) now had reported lesion volume by LST-AI. Only 4 subjects reported "zero" lesion volume with LST-AI.

Overall, there seems to be a significant correlation between the values reported by LST-LGA and LST-AI (rho = 0.5, p < 0.001). Still, LST-AI seems to overestimate the reported lesion volumes of LST-LGA. I am a little bit confused now as to which toolbox we should use for the WMH segmentation.

Any suggestions are welcomed! You can always reach me at pradhanhitesh@cbr-iisc.ac.in for further details and any ideas for validating the toolboxes. Thank you!

twiltgen commented 2 months ago

Hi @pradhanhitesh, thank you for using our tool and reporting your experiences. In general, we found (as reported in our publication) that LST-AI exhibits a much higher sensitivity (and specificity at the same time) than LST-LGA on the data that we used to evaluate our tool. Therefore, the "overestimation of the reported lesion volumes of LST-LGA" can be expected, but mainly because LST-AI detects lesions that were previously not detected by LST-LGA (at least in our experience). Moving forward, we would suggest using LST-AI rather than LST-LGA, which we also discuss and justify in our publication. If, after manually inspecting the images and lesion masks, you feel that many of the lesions segmented by LST-AI are false positives, we would be very grateful to get more detailed insights on these cases so that we can find and address these issues. If this is the case, you can just let us know, and we will then get in touch with you via email.

pradhanhitesh commented 1 month ago

Thank you for getting back @twiltgen. I will be attending AAIC 2024 and would not be able to carry out the proposed actions. However, after coming back from the conference, I will take a look at it and update you as well.