Unable to View Metrics when Training with Single Instance

I have trained a SLEAP model with a single instance using Google Colab. However, training with a single instance does not give me the option to view metrics, such as the error distance, localization error, and object keypoint similarity.

Since the metrics of error distance, localization error, and object keypoint similarity are used to assess the accuracy and validity of models with multiple instances, then I think it would be helpful to include these metrics for models with a single instance. This is the training command for a single instance SLEAP model on Google Colab: !sleap-train single_instance.json filename.pkg.slp.

I thought that I would be able to generated the metrics for a single instance SLEAP model, just as I can for a SLEAP model with multiple instances.

I am unable to load the metrics, as the single instance SLEAP model does not contain the files metrics.train.npz and metrics.val.npz.

I am using SLEAP version 1.3.1 in Google Colab Pro +.

I have included the list of commands I have run in my Google Colab script, which has been adapted from the Google Colab examples on the SLEAP website. I experience an error when I load the last command in this list, which involves a metrics file that does not appear to be generated. I would like SLEAP to generate metrics files for single instance SLEAP models so that I can assess them. Or, can you show me a way to generate figures or other data that can showcase the accuracy and validity of a single instance SLEAP model?

!pip uninstall -y opencv-python opencv-contrib-python
!pip install sleap==1.3.1

from google.colab import drive
drive.mount('/content/drive/')

i``` mport os os.chdir('/content/drive/My Drive/1.28.24_three_chamber')

from google.colab import drive drive.mount('/content/drive')

%load_ext tensorboard %tensorboard --logdir "/content/drive/My Drive/1.28.24_three_chamber" --reload_multifile=true --reload_interval=1 --max_reload_threads=2


`!sleap-train single_instance.json 3.18.23.3.pkg.slp
`
`!ls models
`
`!apt install tree
`
`!tree models/240128_195816.single_instance
`

import sleap import numpy as np import pandas as pd import matplotlib as mpl import matplotlib.pyplot as plt import seaborn as sns

mpl.style.use("seaborn-deep") sleap.versions()

metrics = sleap.load_metrics("models/240128_195816.single_instance", split="val") print("\n".join(metrics.keys()))



Here is the error I receive after running the last command, which begins with metrics. 
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-11-63d4081fb2d8>](https://localhost:8080/#) in <cell line: 1>()
----> 1 metrics = sleap.load_metrics("models/240128_195816.single_instance", split="val")
      2 print("\n".join(metrics.keys()))

1 frames
[/usr/local/lib/python3.10/dist-packages/numpy/lib/npyio.py](https://localhost:8080/#) in load(file, mmap_mode, allow_pickle, fix_imports, encoding, max_header_size)
    403         if hasattr(file, 'read'):
    404             fid = file
--> 405             own_fid = False
    406         else:
    407             fid = stack.enter_context(open(os_fspath(file), "rb"))

FileNotFoundError: [Errno 2] No such file or directory: 'models/240128_195816.single_instance/metrics.val.npz'

talmolab / sleap

Unable to View Metrics when Training with Single Instance #1672