SLEAP centroid-only and pose-only models don't work with Bonsai's 'PredictCentroids' aand 'PredictPoses' nodes, respectively

[jkbhagatio]

Currently, we can export a SLEAP v1.3.1 centroid+centered_instance for inference in Bonsai with the PredictPoseIdentities node using Bonsai's "Bonsai - SLEAP Library v0.2.0".

However, neither the centroid-only nor pose-only model exports work with the respective Bonsai PredictCentroids and PredictPoses nodes.

I used an adaptation of Bruno's code here to manually export the centroid-only and pose-only models (this cannot be done via SLEAP code currently: they only support exports of the stitched models). https://github.com/bruno-f-cruz/Sleap_export-Bonsai/blob/main/sleap_scripts/export_networks.ipynb

Using either SLEAP v1.3.1 or SLEAP v1.2.9, this code was able to extract a 'frozen_graph.pb' file from the model h5 file, but neither of these worked in Bonsai:

• For SLEAP v1.3.1, the error was:

  Object reference not set to an instance of an object

• For SLEAP v1.2.9, the error was:

  Bonsai.WorkflowRuntimeException: 2 root error(s) found.
  (0) NOT_FOUND: No algorithm worked!  Error messages:
   [[{{node StatefulPartitionedCall/StatefulPartitionedCall/centroid_inference_model/centroid_crop/StatefulPartitionedCall/model/stack0_enc0_conv0/Conv2D}}]]
   [[Identity_2/_99]]
  (1) NOT_FOUND: No algorithm worked!  Error messages:
   [[{{node StatefulPartitionedCall/StatefulPartitionedCall/centroid_inference_model/centroid_crop/StatefulPartitionedCall/model/stack0_enc0_conv0/Conv2D}}]]
  0 successful operations.
  0 derived errors ignored. ---> TensorFlow.TFException: 2 root error(s) found.
  (0) NOT_FOUND: No algorithm worked!  Error messages:
   [[{{node StatefulPartitionedCall/StatefulPartitionedCall/centroid_inference_model/centroid_crop/StatefulPartitionedCall/model/stack0_enc0_conv0/Conv2D}}]]
   [[Identity_2/_99]]
  (1) NOT_FOUND: No algorithm worked!  Error messages:
   [[{{node StatefulPartitionedCall/StatefulPartitionedCall/centroid_inference_model/centroid_crop/StatefulPartitionedCall/model/stack0_enc0_conv0/Conv2D}}]]
  0 successful operations.
  0 derived errors ignored.
  at TensorFlow.TFStatus.CheckMaybeRaise(TFStatus incomingStatus, Boolean last)
  at TensorFlow.TFSession.Run(TFOutput[] inputs, TFTensor[] inputValues, TFOutput[] outputs, TFOperation[] targetOpers, TFBuffer runMetadata, TFBuffer runOptions, TFStatus status)
  at TensorFlow.TFSession.Runner.Run(TFStatus status)
  at Bonsai.Sleap.PredictCentroids.<>c__DisplayClass20_1.<Process>b__1(IplImage[] input)
  at System.Reactive.Linq.ObservableImpl.Select`2._.OnNext(TSource value)

[glopesdev]

@jkbhagatio I think this was solved in the last couple of weeks, right? Feel free to close with any additional details. We can also open a different issue for documenting any specific SLEAP training steps that need to be in place for the tracking pipeline to work.