DeepLabCut / napari-deeplabcut

a napari plugin for labeling and refining keypoint data within DeepLabCut projects
GNU Lesser General Public License v3.0
52 stars 22 forks source link

refine_labels on Windows not working after inference on Linux #76

Closed NielsWW closed 1 year ago

NielsWW commented 1 year ago

Hi,

I would like to point out an issue when new videos are analysed on a Linux-driven machine, but outlier frames are extracted and the labels refined on a Windows machine.

I will first describe the workflow I used to train a model with minimal resources:

  1. Label frames on Windows machine.
  2. Port project to Azure compute, which runs Ubuntu.
  3. Create test/train datasets and train a model.
  4. Analyze both videos that were part of the project and new videos.
  5. Port the project and new analyzed videos back to Windows machine.
  6. Extract outlier frames for all videos, both part of the project and the 'new' videos in a folder outside of the project folder.
  7. Refine labels.

For each 'port', I ensured that the project_path was properly set. Now, when I try to save the refined layer, I get an error as the config file cannot be found:

Traceback (most recent call last): File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari_deeplabcut_widgets.py", line 207, in lambda: _save_layers_dialog( File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari_deeplabcut_widgets.py", line 94, in _save_layers_dialog self.viewer.layers.save("", selected=True, plugin="napari-deeplabcut") File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari\components\layerlist.py", line 476, in save return save_layers(path, layers, plugin=plugin, _writer=_writer) File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari\plugins\io.py", line 229, in save_layers _written, writer_name = _write_single_layer_with_plugins( File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari\plugins\io.py", line 442, in _write_single_layer_with_plugins written_paths, writer_name = _npe2.write_layers( File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari\plugins_npe2.py", line 101, in write_layers paths, writer = io_utils.write_get_writer( File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\npe2\io_utils.py", line 115, in write_get_writer return _write(path, layer_data, plugin_name=plugin_name, return_writer=True) File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\npe2\io_utils.py", line 224, in _write res = writer.exec(args=args) File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\npe2\manifest\utils.py", line 62, in exec return self.get_callable(_registry)(*args, **kwargs) File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari_deeplabcut_writer.py", line 65, in write_hdf config = _load_config(os.path.join(project_folder, "config.yaml")) File "C:\Users\NielsW\AppData\Local\miniconda3\envs\bradykinesia\lib\site-packages\napari_deeplabcut_reader.py", line 153, in _load_config with open(config_path) as file: FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/NielsW/Documents/Research Project/Bradykinesia-NielsW-2023-03-08/labeled-data/PD2_4_L_ON\config'

This must be since somewhere the DLCHeader is constructed on Ubuntu and the current method to split the last two directories from the root path relies on the path separators of the currently used OS. I have an easy fix that solves this issue, but I'm not well-versed with GitHub so I'm uncertain how to make a pull request. My proposal is as follows:

in _writer.py, line 65 was: project_folder = root.rsplit(os.sep, 2)[0]

I propose to replace it with: project_folder = os.path.dirname(os.path.dirname(root))

Maybe I'm overlooking some problems this might create, or I did something wrong when trying to port the project. My apologies if this issue is already addressed. I use napari-deeplabcut 0.0.9

jeylau commented 1 year ago

Hello @NielsWW, thanks for reporting this bug. I had encountered it about 2 months ago and addressed it with https://github.com/DeepLabCut/napari-deeplabcut/commit/a65f78e578549b02d11fa0dd3afd94e9d6390ce1. Please run pip install -U napari-deeplabcut to make sure you get the latest version and the fix ☺️

NielsWW commented 1 year ago

Hi @jeylau, thanks, will do!