AllenInstitute / npc_sessions

Tools for accessing and packaging data from behavior and epyhys sessions from the Mindscope Neuropixels team, in the cloud.
1 stars 1 forks source link

handle missing probes from sorting pipeline #102

Closed bjhardcastle closed 4 months ago

bjhardcastle commented 4 months ago
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/npc_ephys/units.py", line 269, in make_units_table_from_spike_interface_ks25
    _ = future.result()
  File "/opt/conda/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/opt/conda/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/opt/conda/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/conda/lib/python3.9/site-packages/npc_ephys/units.py", line 163, in _device_helper
    spike_interface_data.electrode_locations_xy(electrode_group_name)
  File "/opt/conda/lib/python3.9/site-packages/npc_ephys/spikeinterface.py", line 458, in electrode_locations_xy
    self.sorting_json(probe)["annotations"]["__sorting_info__"]["recording"][
  File "/opt/conda/lib/python3.9/site-packages/npc_ephys/spikeinterface.py", line 419, in sorting_json
    self.get_correct_path(self.postprocessed(probe), "sorting.json")
  File "/opt/conda/lib/python3.9/site-packages/npc_ephys/spikeinterface.py", line 141, in get_correct_path
    path = npc_io.from_pathlike("/".join(str(path) for path in path_components))
  File "/opt/conda/lib/python3.9/site-packages/npc_io/file_io.py", line 65, in from_pathlike
    raise FileNotFoundError(
FileNotFoundError: In attempting to handle a path containing '#', we couldn't find s3://codeocean-s3datasetsbucket-1u41qdg42ur9/921d311f-4384-4230-b2cd-3d9b58570942/postprocessed/postprocessed-sorting_experiment1_Record Node 103#Neuropix-PXI-100.ProbeE-AP_recording1/sorting.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/scripts/write_sessions_to_cache.py", line 88, in <module>
    main()
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/scripts/write_sessions_to_cache.py", line 84, in main
    write_sessions_to_cache(**kwargs)  # type: ignore[misc, arg-type]
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/scripts/write_sessions_to_cache.py", line 73, in write_sessions_to_cache
    helper(
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/scripts/write_sessions_to_cache.py", line 27, in helper
    npc_sessions.write_all_components_to_cache(
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/utils/cache.py", line 248, in write_all_components_to_cache
    component = _get_nwb_component(session, component_name)
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/utils/cache.py", line 96, in _get_nwb_component
    if session.analysis:
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/sessions.py", line 742, in analysis
    for module in self._analysis:
  File "/opt/conda/lib/python3.9/site-packages/npc_io/cached_property.py", line 78, in __get__
    val = self.func(instance)
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/sessions.py", line 754, in _analysis
    modules.append(self.all_spike_histograms)
  File "/opt/conda/lib/python3.9/site-packages/npc_io/cached_property.py", line 78, in __get__
    val = self.func(instance)
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/sessions.py", line 2634, in all_spike_histograms
    for probe in sorted(set(self.units.electrode_group_name[:])):
  File "/opt/conda/lib/python3.9/site-packages/npc_io/cached_property.py", line 78, in __get__
    val = self.func(instance)
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/sessions.py", line 1436, in units
    for column in self._units.columns:
  File "/opt/conda/lib/python3.9/site-packages/npc_io/cached_property.py", line 78, in __get__
    val = self.func(instance)
  File "/opt/conda/lib/python3.9/site-packages/npc_sessions/sessions.py", line 1375, in _units
    units=npc_ephys.make_units_table_from_spike_interface_ks25(
  File "/opt/conda/lib/python3.9/site-packages/npc_ephys/units.py", line 280, in make_units_table_from_spike_interface_ks25
    raise RuntimeError(
RuntimeError: Error fetching units for 668755_2023-08-28 - see original exception above/below
bjhardcastle commented 4 months ago

Same issue for all probes - sorted output must be bad

bjhardcastle commented 4 months ago

only probes A and E are bad: now excluded from session