Closed mariapfj closed 6 months ago
Thanks for reporting this @mariapfj
Will push a fix tomorrow morning. The problem is that the widgets part is not tested because it requires user input, so there was probably an unspotted naming issue
side note: tried running kilosort on the registered (no depth) data, which failed. Pasting the entire error below as there's a couple of things that might need debugging (dubious ownership?)
Spike sorting with kilosort2 using installed sorter fatal: detected dubious ownership in repository at '/fp/projects01/ec109/spikesorters/Kilosort2' To add an exception for this directory, call:
git config --global --add safe.directory /fp/projects01/ec109/spikesorters/Kilosort2
Usage: which [options] [--] COMMAND [...] Write the full path of COMMAND(s) to standard output.
--version, -[vV] Print version and exit successfully. --help, Print this help and exit successfully. --skip-dot Skip directories in PATH that start with a dot. --skip-tilde Skip directories in PATH that start with a tilde. --show-dot Don't expand a dot to current directory in output. --show-tilde Output a tilde for HOME directory for non-root. --tty-only Stop processing options on the right if not on tty. --all, -a Print all matches in PATH, not just the first --read-alias, -i Read list of aliases from stdin. --skip-alias Ignore option --read-alias; don't read stdin. --read-functions Read shell functions from stdin. --skip-functions Ignore option --read-functions; don't read stdin.
Recommended use is to write the output of (alias; declare -f) to standard input, so that which can show aliases and shell functions. See which(1) for examples.
If the options --read-alias and/or --read-functions are specified then the output can be a full alias or function definition, optionally followed by the full path of each command used inside of those.
Report bugs to which-bugs@gnu.org. ERROR: unable to process 158-290224-1 Traceback (most recent call last): File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/expipe_plugin_cinpla/scripts/process.py", line 218, in process_ecephys sorting = ss.run_sorter( ^^^^^^^^^^^^^^ File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/runsorter.py", line 174, in run_sorter return run_sorter_local(**common_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/runsorter.py", line 224, in run_sorter_local SorterClass.run_from_folder(output_folder, raise_error, verbose) File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 293, in run_from_folder raise SpikeSortingError( spikeinterface.sorters.utils.misc.SpikeSortingError: Spike sorting error trace: Traceback (most recent call last): File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 258, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/external/kilosortbase.py", line 217, in _run_from_folder raise Exception(f"{cls.sorter_name} returned a non-zero exit code") Exception: kilosort2 returned a non-zero exit code
Spike sorting failed. You can inspect the runtime trace in /projects/ec109/maria/novel-env-nwb/actions/158-290224-1/data/spikeinterface/kilosort2/spikesorting/spikeinterface_log.json.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/expipe_plugin_cinpla/widgets/process.py", line 296, in on_run process.process_ecephys( File "/projects/ec109/conda-envs/cinpla/lib/python3.11/site-packages/expipe_plugin_cinpla/scripts/process.py", line 234, in process_ecephys raise Exception(f"Spike sorting failed:\n\n{e}") Exception: Spike sorting failed:
Spike sorting error trace: Traceback (most recent call last): File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/basesorter.py", line 258, in run_from_folder SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose) File "/fp/homes01/u01/ec-mariapfj/.local/lib/python3.11/site-packages/spikeinterface/sorters/external/kilosortbase.py", line 217, in _run_from_folder raise Exception(f"{cls.sorter_name} returned a non-zero exit code") Exception: kilosort2 returned a non-zero exit code
Spike sorting failed. You can inspect the runtime trace in /projects/ec109/maria/novel-env-nwb/actions/158-290224-1/data/spikeinterface/kilosort2/spikesorting/spikeinterface_log.json.
@mariapfj
I think that the problem of registering the depth whenr registering an action should be fixed with #64 and surgeries get registered properly
@mariapfj
The registration is fixed on Educloud. To run KS2, you need to run this command (once):
git config --global --add safe.directory /fp/projects01/ec109/spikesorters/Kilosort2
@mariapfj can you confirm this is fixed?
@alejoe91 the cinpla kernel is back to crashing every time we run a simple command, if you sort this i'll get right to testing out the registration
Hey, I can't access educloud at the moment..what is the error? still some permission issues?
the cinpla kernel crashes when running any cell, even a simple print() function. we've had this issue previously when you were just setting up the kernel, but not sure if it's a permission error or something else
we've also had issues simply logging onto educloud on demand this week, not sure if that's connected in any way
hold up I found it: Permission denied: '/fp/projects01/ec109/conda-envs/cinpla/lib/python3.11/site-packages/astropy-6.0.0.dist-info'
thanks, I'll check
@lepmik did you modify the environment?
Can you give group permissions?
hold up I found it: Permission denied: '/fp/projects01/ec109/conda-envs/cinpla/lib/python3.11/site-packages/astropy-6.0.0.dist-info'
Same error on my side
Sorry, I forgot... Should work now
seems like its partly working now, was able to register a surgery and adjustment, but got an error when registering a recording specifically when I wanted to register the depth/adjustment (works fine without), see error below. tiny comment, I noticed the depth adjustment and annotation pages have no log to show for, which is a nice feature to have :)
AttributeError Traceback (most recent call last)
File /cluster/software/EL9/easybuild/software/jupyter-server/2.7.2-GCCcore-12.3.0/lib/python3.11/site-packages/ipywidgets/widgets/widget_output.py:103, in Output.capture.
File /fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/widgets/register.py:104, in register_openephys_view.
File /fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/register.py:142, in register_openephys_recording(project, action_id, openephys_path, probe_path, depth, overwrite, include_events, entity_id, user, session, location, message, tags, delete_raw_data, correct_depth_answer, register_depth) 139 action.location = location 141 if register_depth: --> 142 correct_depth = utils_register_depth(project=project, action=action, depth=depth, answer=correct_depth_answer) 143 if not correct_depth: 144 print("Aborting registration!")
File /fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/utils.py:132, in register_depth(project, action, depth, answer, overwrite) 130 adjustdate = None 131 else: --> 132 curr_depth, adjustdate = get_depth_from_adjustment(project, action, action.entities[0]) 133 print("Adjust date time: {}\n".format(adjustdate)) 134 if curr_depth is None:
File /fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/utils.py:118, in get_depth_from_adjustment(project, action, entity_id) 116 regdate = action.datetime 117 adjustdates = adjusts.keys() --> 118 adjustdate = min(adjustdates, key=lambda x: deltadate(x, regdate)) 119 return adjusts[adjustdate]["depth"].contents, adjustdate
File /fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/utils.py:118, in get_depth_from_adjustment.
File /fp/projects01/ec109/software/expipe-plugin-cinpla/src/expipe_plugin_cinpla/scripts/utils.py:50, in deltadate(adjustdate, regdate) 49 def deltadate(adjustdate, regdate): ---> 50 delta = regdate - adjustdate if regdate > adjustdate else datetime.timedelta.max 51 return delta
AttributeError: type object 'datetime.datetime' has no attribute 'timedelta'
Will push a fix soon!
@mariapfj the error should be fixed and I added logs for the missing tabs!
You might need to restart the kernel ;)
Tried to register actions in educloud, but it couldn't find depth measurements (would cause issues in downstream scripts and is used to separate different experiments in the same animal). The raw data and the probe are both available in my folder, as is entity data from the nwb conversion. Surgery and adjustment folders cause issues when converting the project, but I tried copying them into the new action folder just in case it was able to get depth data from there (it was not). It seems to work fine when registering without depth.
I then tried to register the surgery and the entity, both giving similar error messages (see under). I've also tried converting the actions directly from exdir but unsuccessfully (with a previous version of expipe-plugin-cinpla the latter returned av error message about 'processing' not existing, but current version simply skips unsuccessful attempts). Any clue what's causing these errors?