Open benlipkin opened 2 years ago
This looks like a case of something being lost while caching or not recovered from cache at the time of load_cache
I'm able to replicate this stochastically; need to test Dataset caching behavior to see whether everything is being recovered or not.
something's going wrong while deserializing the cached Dataset xarray. All metadata is recovered but the data is converted to NaNs.
In [33]: e = xr.open_zarr('/home/aalok/.cache/langbrainscore/Dataset/(Dataset?dataset_name=pereira2018_mean_froi_Lang)/_xr_obj.xr') In [34]: e.data Out[34]: <xarray.DataArray 'data' (sampleid: 627, neuroid: 108, timeid: 1)> array([[[nan], [nan], ..., [nan], [nan]], [[nan], [nan], ..., [nan], [nan]], ..., [[nan], [nan], ..., [nan], [nan]], [[nan], [nan], ..., [nan], [nan]]]) Coordinates: experiment (sampleid) <U16 ... * neuroid (neuroid) int64 0 1 2 3 4 5 6 7 ... 261 262 263 264 265 266 267 passage (sampleid) <U12 ... roi (neuroid) <U16 ... * sampleid (sampleid) int64 0 1 2 3 4 5 6 7 ... 620 621 622 623 624 625 626 session (neuroid) <U17 ... stimulus (sampleid) <U119 ... subject (neuroid) int64 ... * timeid (timeid) int64 0 Attributes: measurement: fmri modality: text source: /home/aalok/code/langbrainscore/data/Pereira_FirstSession_T...
tried running
test_mean_froi_pereira2018_firstsessions.py
runs fine at first, but the second time, when loading from cache, I see this error:
If I comment out this section
and just recalculate the dataset instead:
then the error goes away.
worth looking into.