Just for the record, I'm documenting here an idea from @plbenveniste raised during the 2024-11-07 NeuroPoly lab meeting.
Idea: instead of having a two-model approach (model 1: lesion and cord segmentation and morphometrics calculation, model 2: patients' outcome predictive model based on MRI morphometrics and clinical scores), we could train a single multimodal model (trained simultaneously on MRI scans and text clinical data).
TODO:
add refs to some recent foundation multimodal models
Just for the record, I'm documenting here an idea from @plbenveniste raised during the 2024-11-07 NeuroPoly lab meeting.
Idea: instead of having a two-model approach (model 1: lesion and cord segmentation and morphometrics calculation, model 2: patients' outcome predictive model based on MRI morphometrics and clinical scores), we could train a single multimodal model (trained simultaneously on MRI scans and text clinical data).
TODO: