Error running job (pjh): Simple selection can't process <HDF5 dataset "electrodes": shape (32,), type "<i8">

[khl02007]

We get the following message on sortingview backend:

Error in task recording_info.3: Error running job (pjh): Simple selection can't process <HDF5 dataset "electrodes": shape (32,), type "<i8">
HITHER JOBS: 0 pending; 0 queued; 0 running; 101 finished; 18 errored; 32 cache hit

also see that not every task function is registered

using sortingview backend version 0.6.5

[magland]

Could you share a link to the workspace where this error is occurring?

Also 18/25 is the expected number of registered task functions when only the new backend is running.

[acomrie]

I do not know which workspace @khl02007 was using when he got this error, but here is a workspace that was giving me trouble (and which I asked Kyu about before this issue was reported): https://sortingview.vercel.app/workspace/recording/R-5772b0fe308b?workspace=e127db62d41648dd9cb65b1cbf294da37fb65ba643b6938e3a46d182762bc208&channel=franklab

[khl02007]

just for note: during meeting today we thought this was probably due to pynwb version (v 1.5 works but 2.0 doesn't). Actually, I just tried running everything in a copy of the old environment but pynwb updated to v2.0. And it works - this error doesn't get thrown. So the cause is still unclear, but it's good that it got resolved.

[acomrie]

Thanks @khl02007 et al; the workspaces that were giving me trouble are also opening like normal now!

[magland]

Great. I guess it would be good to get to the bottom of the issue. @khl02007 could you please post the output of pip list on the problematic conda environment? (also list the conda packages i guess)

[khl02007]

@magland In kachery-env2 (the cloned kachery env with pynwb updated to 2.0) but not in figurl env:

aiohttp    3.7.4.post0
aiohttp-cors    0.7.0
async-timeout    3.0.1
cffi    1.14.5
chardet    4.0.0
google-api-core    1.30.0
google-cloud-core    1.7.1
google-cloud-storage    1.40.0
google-crc32c    1.1.2
google-resumable-media    1.3.1
googleapis-common-protos    1.53.0
kachery    0.6.7
kachery-daemon    1.0.27
labbox    0.1.35
labbox-ephys    0.7.2
multidict    5.1.0
networkx    2.5.1
packaging    20.9
paho-mqtt    1.5.1
protobuf    3.17.3
pycparser    2.20
pyparsing    2.4.7
scikit-learn    0.24.2
spikefeatures    0.1.2
spikemetrics    0.2.4
spiketoolkit    0.7.5
threadpoolctl    2.1.0
typing-extensions    3.10.0.0
websockets    9.1
yarl    1.6.3

In figurl env but not in kachery-env2:

charset-normalizer    2.0.4

In both but different versions (kachery-env2 vs. figurl env)

decorator    4.4.2  vs.  5.0.9
google-auth    1.32.0  vs.  2.0.1
h5py    2.10.0  vs.  3.4.0
idna    2.10  vs.  3.2
ipython    7.25.0  vs.  7.27.0
numpy    1.20.3  vs.  1.21.2
pandas    1.2.5  vs.  1.3.2
pip    21.2.4  vs.  21.0.1
prompt-toolkit    3.0.19  vs.  3.0.20
Pygments    2.9.0  vs.  2.10.0
python-dateutil    2.8.1  vs.  2.8.2
requests    2.25.1  vs.  2.26.0
ruamel.yaml    0.17.10  vs.  0.17.16
ruamel.yaml.clib    0.2.4  vs.  0.2.6
scipy    1.7.0  vs.  1.7.1
setuptools    57.0.0  vs.  52.0.0.post20210125
simplejson    3.17.2  vs.  3.17.5
spikeextractors    0.9.6  vs.  0.9.7
tqdm    4.61.1  vs.  4.62.2
websocket-client    1.1.0  vs.  1.2.1
wheel    0.36.2  vs.  0.37.0

I tried updated h5py to 3.4; still works, so that's probably not it. Could it be one of those packages related to google cloud storage?

[magland]

Thanks @khl02007 , it's sort of mysterious. I'm closing this for now, but it's good to have that record of which packages are installed in the working environment.