Difference in predicted age between standard and dockerised version of scipts

[Calkinch]

Hi, When running the docker version of the script I am getting predicted ages that are significantly younger than when running the standard script on the same data. For example with the standard script the predicted age was 47.37, but for the same data with the docker script the predicted age was 37.54. I was expecting some minor variations of course, but this seems too large. Has anyone experienced this? If so what might be causing it, and do you know how I could I rectify this to align the predictions.

[james-cole]

Hi there,

Thanks for flagging this up. I’m not aware that anyone has reported this previously, and it’s a bit surprising as the model is the same in both versions. One possibility is that there’s an issue with the versioning of one of the packages, and it’s not loading properly, thus ends up just predicting the mean (~37 years is close to the mean age of the training dataset). Was this just in one participant? Or does this happen to everyone?

Best wishes, James

From: Calkinch @.> Date: Tuesday, 2 July 2024 at 11:56 am To: james-cole/brainageR @.> Cc: Subscribed @.***> Subject: [james-cole/brainageR] Difference in predicted age between standard and dockerised version of scipts (Issue #15)

⚠ Caution: External sender

Hi, When running the docker version of the script I am getting predicted ages that are significantly younger than when running the standard script on the same data. For example with the standard script the predicted age was 47.37, but for the same data with the docker script the predicted age was 37.54. I was expecting some minor variations of course, but this seems too large. Has anyone experienced this? If so what might be causing it, and do you know how I could I rectify this to align the predictions.

— Reply to this email directly, view it on GitHubhttps://github.com/james-cole/brainageR/issues/15, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCUV6DQLELOMWVDTUGRTSDZKKBPLAVCNFSM6AAAAABKHJPO2GVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4DKOJTGQYDINQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Calkinch]

Hi James, Thank you for such a swift reply.

I have run both scripts (Standard and Docker) on 23 participants. The docker script produced a younger prediction that the standard script every time, a small number of them returned predictions that where closer to each other (within 5 years) but for the majority of them the docker script returned a significantly younger brain age. The biggest difference between the predicted ages of the two scripts, for the same input image was 24.91 years (standard predicted 62.01 and docker predicted 37.09). Best wishes, Cal

[james-cole]

Hi Cal,

Thanks for the further information. Can I ask whether the predicted age distribution for the 23 participants on the Docker version is narrowly around ~37 years? If so, it suggests something has gone awry with that version and it is just predicted the mean? Did you look at the mean and SD for each version?

From: Calkinch @.> Date: Tuesday, 2 July 2024 at 12:18 pm To: james-cole/brainageR @.> Cc: Cole, James @.>, Comment @.> Subject: Re: [james-cole/brainageR] Difference in predicted age between standard and dockerised version of scipts (Issue #15)

⚠ Caution: External sender

Hi James, Thank you for such a swift reply.

I have run both scripts (Standard and Docker) on 23 participants. The docker script produced a younger prediction that the standard script every time, a small number of them returned predictions that where closer to each other (within 5 years) but for the majority of them the docker script returned a significantly younger brain age. The biggest difference between the predicted ages of the two scripts, for the same input image was 24.91 years (standard predicted 62.01 and docker predicted 37.09). Best wishes, Cal

— Reply to this email directly, view it on GitHubhttps://github.com/james-cole/brainageR/issues/15#issuecomment-2202816548, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCUV6BTAJHS4IB5B4BFZMTZKKD7HAVCNFSM6AAAAABKHJPO2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBSHAYTMNJUHA. You are receiving this because you commented.Message ID: @.***>

[Calkinch]

Hi James, Yes, I have the mean and SD. The standard script is M=49.7, SD=8.42, and the docker script is M=34.5, SD=7.73. Also the two scripts predictions correlate to r=0.75.

Best wishes, Cal

[james-cole]

Hi Cal,

OK, that’s interesting. Clearly something isn’t working with one or both of the versions. Are these healthy individuals and do you know the correlations with chronological by version? Also, did you QC the segmentations? These should be the same, but could be a cause of the discrepancy if SPM12 segment is failing for one and not the other.

Final thought, what type of scanner are you using? This shouldn’t cause a discrepancy, but could impair generalisability if it’s one not represented in the training dataset.

Best wishes, James

From: Calkinch @.> Date: Tuesday, 2 July 2024 at 4:28 pm To: james-cole/brainageR @.> Cc: Cole, James @.>, Comment @.> Subject: Re: [james-cole/brainageR] Difference in predicted age between standard and dockerised version of scipts (Issue #15)

⚠ Caution: External sender

Hi James, Yes, I have the mean and SD. The standard script is M=49.7, SD=8.42, and the docker script is M=34.5, SD=7.73. Also the two scripts predictions correlate to r=0.75.

Best wishes, Cal

— Reply to this email directly, view it on GitHubhttps://github.com/james-cole/brainageR/issues/15#issuecomment-2203532659, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCUV6B3MKN3HNO3J4O7NI3ZKLBK5AVCNFSM6AAAAABKHJPO2GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBTGUZTENRVHE. You are receiving this because you commented.Message ID: @.***>