Error when saving OnACID object

[StoyoKaramihalev]

Please fill in the following for any issues

Your setup:

1. Operating System (Linux, MacOS, Windows): MacOS
2. Hardware type (x86, ARM..) and RAM: M1, 64GB RAM
3. Python Version (e.g. 3.9): 3.11.6
4. Caiman version (e.g. 1.9.12): 1.9.16
5. Which demo exhibits the problem (if applicable): NA
6. How you installed Caiman (pure conda, conda + compile, colab, ..): pure conda
7. Details:

OnACID.save produces a value error related to corr_img

---

ValueError Traceback (most recent call last) Untitled-1.ipynb Cell 7 line 3 1 #%% Save 2 print('Saving CNMF result') ----> 3 cnm.save('cnmf_online_res.hdf5')

File ~/opt/anaconda3/envs/caiman/lib/python3.11/site-packages/caiman/source_extraction/cnmf/online_cnmf.py:1079, in OnACID.save(self, filename) 1070 """save object in hdf5 file format 1071 1072 Args: 1073 filename: str 1074 path to the hdf5 file containing the saved object 1075 """ 1077 if '.hdf5' in filename: 1078 # keys_types = [(k, type(v)) for k, v in self.dict.items()] -> 1079 save_dict_to_hdf5(self.dict, caiman.paths.fn_relocated(filename)) 1080 else: 1081 raise Exception("Unsupported file extension")

File ~/opt/anaconda3/envs/caiman/lib/python3.11/site-packages/caiman/utils/utils.py:435, in save_dict_to_hdf5(dic, filename, subdir) 426 ''' Save dictionary to hdf5 file 427 Args: 428 dic: dictionary (...) 431 file name to save the dictionary to (in hdf5 format for now) 432 ''' 434 with h5py.File(filename, 'w') as h5file: --> 435 recursively_save_dict_contents_to_group(h5file, subdir, dic)

File ~/opt/anaconda3/envs/caiman/lib/python3.11/site-packages/caiman/utils/utils.py:535, in recursively_save_dict_contents_to_group(h5file, path, dic) 533 h5file[path + key] = np.array(item) 534 elif type(item).name in ['CNMFParams', 'Estimates']: # parameter object --> 535 recursively_save_dict_contents_to_group(h5file, path + key + '/', item.dict) 536 else: 537 raise ValueError(f"Cannot save {type(item)} type for key '{key}'.")

File ~/opt/anaconda3/envs/caiman/lib/python3.11/site-packages/caiman/utils/utils.py:517, in recursively_save_dict_contents_to_group(h5file, path, dic) 515 h5file[path + key] = item 516 if not np.array_equal(h5file[path + key][()], item): --> 517 raise ValueError(f'Error while saving ndarray {key} of dtype {item.dtype}') 518 # save dictionaries 519 elif isinstance(item, dict):

ValueError: Error while saving ndarray corr_img of dtype float32

[EricThomson]

Thanks for bringing this up. We will check out and see if we can reproduce.

[EricThomson]

@StoyoKaramihalev sorry for the delay getting to this. I'm in the onacid demo and have run the fit function, and just ran:

save_path =  r'saved_onacid_results.hdf5'  # or add full/path/to/file.hdf5
cnm.save(save_path)

For me it ran without error. Could you let me know steps to reproduce the problem with saving onacid object in your use case?

[FlorianNeurosci]

Hi, i ran into the same error when trying to save my onacid object, also during saving the corr image. As I dont know what the best way to errorhunt this is, I saved the corr image as a npy and uploaded it in a zip here (as I am not allowed to upload .npy). corr_img.zip If I run this snippet (which I took from the recursive save function), the comparison comes out as False, which leads to the error on the bottom: Hope this helps and I am happy for all ideas how to solve this!

code snippet to reproduce error:

test = np.load('corr_img.npy')
with h5py.File('test', 'w') as h5file:
    h5file['test'] = cnm_online.estimates.corr_img
with h5py.File('test', 'r') as h5file:
    print(h5file['test'][()])
    print(np.array_equal(h5file['test'][()], cnm_online.estimates.corr_img))
    print(np.array_equal(h5file['test'][()], test))

all comparisons evaluate to False for me

error message from caiman

 File [~/mambaforge/envs/calciumEnv/lib/python3.10/site-packages/caiman/utils/utils.py:532](http://localhost:8083/home/fs539/mambaforge/envs/calciumEnv/lib/python3.10/site-packages/caiman/utils/utils.py#line=531), in recursively_save_dict_contents_to_group(h5file, path, dic)
    530     h5file[path + key] = np.array(item)
    531 elif type(item).__name__ in ['CNMFParams', 'Estimates']: #  parameter object
--> 532     recursively_save_dict_contents_to_group(h5file, path + key + '[/](http://localhost:8083/)', item.__dict__)
    533 else:
    534     raise ValueError(f"Cannot save {type(item)} type for key '{key}'.")

File [~/mambaforge/envs/calciumEnv/lib/python3.10/site-packages/caiman/utils/utils.py:514](http://localhost:8083/home/fs539/mambaforge/envs/calciumEnv/lib/python3.10/site-packages/caiman/utils/utils.py#line=513), in recursively_save_dict_contents_to_group(h5file, path, dic)
    512         h5file[path + key] = item
    513     if not np.array_equal(h5file[path + key][()], item):
--> 514         raise ValueError(f'Error while saving ndarray {key} of dtype {item.dtype}')
    515 # save dictionaries
    516 elif isinstance(item, dict):

ValueError: Error while saving ndarray corr_img of dtype float32

[EricThomson]

Thanks will look into it.