KeyError: 'grand_mean'

[pesojka]

Hello, there seems to be a bug in the applyModelNIFTIs (bellow). There is no 'grand_mean' key in the model dict, only 'stand_mean', but applyModelOne within harmonizationApply tries to compute 'stand_mean' from 'grand_mean' which is missing in the model.

KeyError Traceback (most recent call last) Cell In[6], line 2 1 my_model = nh.loadHarmonizationModel('MY_MODEL') ----> 2 applyModelNIFTIs(covars, my_model, nifti_list, 'thresholded_mask.nii.gz')

File ~/.local/lib/python3.10/site-packages/neuroHarmonize/harmonizationNIFTI.py:153, in applyModelNIFTIs(covars, model, paths, mask_path) 151 header = nifti.header 152 # nifti_array_adj = applyModelOne(nifti_array, covarsSel, model) --> 153 nifti_array_adj, nifti_array_stand_mean = applyModelOne(nifti_array, covarsSel, model, True) #return stand_mean as well 154 nifti_out = nifti_mask.astype(float).copy() 155 nifti_out[nifti_mask] = nifti_array_adj[0, :]

File ~/.local/lib/python3.10/site-packages/neuroHarmonize/harmonizationApply.py:199, in applyModelOne(data, covars, model, return_stand_mean) 197 ### from neuroCombat implementation: 198 B_hat = model['B_hat'] --> 199 grand_mean = model['grand_mean'] 200 var_pooled = model['var_pooled'] 202 stand_mean = np.dot(grand_mean.T.reshape((len(grand_mean), 1)), np.ones((1, n_sample)))

KeyError: 'grand_mean'

[rpomponio]

Thanks for identifying this bug. It is related to #46.

I will work on patching this in the next 48 hours.

[rpomponio]

@pesojka I just pushed a new version to GitHub in attempts to fix this bug.

Would you mind testing your same code with the latest development version? You will need to install from GitHub:

pip install git+https://github.com/rpomponio/neuroHarmonize --upgrade

Let me know if this works. If so, I will distribute the current version on the Packaging Index.

[pesojka]

Hello, thank you for the swift reply!

Now I am getting following error (same code with the latest version):

---

ValueError Traceback (most recent call last) Cell In[7], line 2 1 my_model = nh.loadHarmonizationModel('MY_MODEL') ----> 2 applyModelNIFTIs(covars, my_model, nifti_list, 'thresholded_mask.nii.gz')

File ~/.local/lib/python3.10/site-packages/neuroHarmonize/harmonizationNIFTI.py:153, in applyModelNIFTIs(covars, model, paths, mask_path) 151 header = nifti.header 152 # nifti_array_adj = applyModelOne(nifti_array, covarsSel, model) --> 153 nifti_array_adj, nifti_array_stand_mean = applyModelOne(nifti_array, covarsSel, model, True) #return stand_mean as well 154 nifti_out = nifti_mask.astype(float).copy() 155 nifti_out[nifti_mask] = nifti_array_adj[0, :]

File ~/.local/lib/python3.10/site-packages/neuroHarmonize/harmonizationApply.py:206, in applyModelOne(data, covars, model, return_stand_mean) 204 tmp = copy.deepcopy(design_i) 205 tmp[:,range(0,n_batch)] = 0 --> 206 mod_mean = np.transpose(np.dot(tmp, B_hat)) 208 s_data = ((X- stand_mean - mod_mean) / np.dot(np.sqrt(var_pooled), np.ones((1, n_sample)))) 210 if sum(isTrainSite)==0:

File <__array_function__ internals>:180, in dot(*args, **kwargs)

ValueError: shapes (1,5) and (6,714401) not aligned: 5 (dim 1) != 6 (dim 0)

[rpomponio]

I can confirm I am getting the same error, but I have not identified the source of the bug.

Will be working on this and try to patch it by the end of this week.

You could try installing an earlier version (2.3.x) if you need this utility sooner.

[rpomponio]

@pesojka I think I have fixed it. Please try again when you have a chance and let me know what error messages (if any) you get.

Thanks!

[pesojka]

Version 2.4.4. works with no error! Thank you for the fix!