search

phys201 / Hamiltonian-photonic-crystal

Build the Hamiltonian model to reproduce the band structure measured on photonic crytal
GNU General Public License v3.0
0 stars 1 forks source link

Couldn't load data with pickle #26

Closed turnerwoody1 closed 1 year ago

turnerwoody1 commented 1 year ago

I could not load in the expdata when running the documentation notebook. Numpy seems to think the data is pickled but when I allow pickled data it still throws the following error: UnpicklingError Traceback (most recent call last) File ~/opt/anaconda3/envs/phys201/lib/python3.11/site-packages/numpy/lib/npyio.py:441, in load(file, mmap_mode, allow_pickle, fix_imports, encoding, max_header_size) 440 try: --> 441 return pickle.load(fid, **pickle_kwargs) 442 except Exception as e:

UnpicklingError: invalid load key, 'H'.

The above exception was the direct cause of the following exception:

UnpicklingError Traceback (most recent call last) Cell In[16], line 2 1 # load processed data from npy ----> 2 exp_data0 = np.load('cut_expdata_GM0p05.nc', allow_pickle=True) 3 # 4 a = 1000e-9 # lattice constant of phc

File ~/opt/anaconda3/envs/phys201/lib/python3.11/site-packages/numpy/lib/npyio.py:443, in load(file, mmap_mode, allow_pickle, fix_imports, encoding, max_header_size) 441 return pickle.load(fid, **pickle_kwargs) 442 except Exception as e: --> 443 raise pickle.UnpicklingError( 444 f"Failed to interpret file {file!r} as a pickle") from e

UnpicklingError: Failed to interpret file 'cut_expdata_GM0p05.nc' as a pickle

My understanding of pickle is that there can be issues if different users are using different versions of pickle, or simply if their installations differ. You might want to consider using a more consistent way to save the data, like an ordinary numpy array.

turnerwoody1 commented 1 year ago

Loading the data with the io.io function later does work. You should clean up the documentation notebook to be up to date with any changes to your package's functionality.

vnmanoharan commented 1 year ago

Agreed - there are several problems with the loading here. The filename is wrong in multiple places in the notebook, and the note to install netCDF4 via pip is incorrect; you should not need any extra packages to load a netCDF file into xarray. See also #18

erandichavez commented 1 year ago

I also could not load in the data. Perhaps saving the data as a csv file and using Pandas to load it could be easier than .npy. I at least have not had issues with this

thaodinh98 commented 1 year ago

We looked into these issues and fix our io_data.py as well as the data formatting. Loading data should not have problem now.