the exported .pb file folde

[Enricocloud]

Sorry, I cannot find this file folder, The Sleap software should generate it, or shoudl create it somehowBecausese you say: "To use the Predict operators, you will need to provide the ModelFileName to the exported .pb file folder containing your pretrained SLEAP model, along with the corresponding PoseConfigFileName to the training_config.json file." I have the .json file but not the .pb

Thanks

[talmo]

Hi @Enricocloud,

You can generate this pb file by using the sleap-export command. Here's more info: https://sleap.ai/guides/cli.html#sleap-export

Cheers,

Talmo

[Enricocloud]

it works, but now the console gives me this error "

TypeError: Exception encountered when calling layer "bottom_up_inference_layer_1" (type BottomUpInferenceLayer).

'Tensor' object cannot be interpreted as an integer

Call arguments received: • args=({'image': 'tf.Tensor(shape=(None, 560, 1440, 1), dtype=uint8)'},) • kwargs=<class 'inspect._empty'> " I try installing the TF==2.8.0 but it doesn't work.

Thanks

[talmo]

Hi @Enricocloud, yep, so we currently only support export of top-down, ID with top-down, centroid and single instance model's unfortunately. You can try retraining on your labels with a top-down model and giving this another go.

[Enricocloud]

ok I created the top-down model and the exported file, but now I get this error, I download all the tools and the DLL and put them on the extension folder, so I don't understand what this error is " error: Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice 2023-06-05 14:00:24.181120: W tensorflow/core/framework/op_kernel.cc:1768] UNKNOWN: JIT compilation failed. "

[bruno-f-cruz]

What is the model of your GPU and what driver version do you have installed? What version of CUDA and cuDNN have you installed? which Tensorflow release did you download?

[Enricocloud]

I'm working on an Nvidia RTX Quadro 4000, I downloaded the latest driver for Win 11, the CUDA v11.3, and cuDNN 8.2 I downloaded the TensorFlow 2.10 for GPU only.

[bruno-f-cruz]

Is this error coming from Bonsai or Python? Can you make sure that everything you are using is compatible across the board?

e.g. https://transang.me/cuda-cudnn-driver-gcc-tensorflow-python-version-compatibility-charts/

Otherwise, I am not sure that, after following the installation steps I ever came across that error.

[Enricocloud]

Thanks for all the reply, I check all the compatibility of the environment, I think the error could be by bonsai. Here it the screen of all the error

[bruno-f-cruz]

What do you mean by environment? Bonsai doesn't need to have any Python installed to run. You must install the dependencies as instructed in the tutorial in order for the inference backend to work.

What does your project Extensions folder look like? The error message leads me to believe that some dependency was just not correctly installed. I would probably just try to follow the instructions once again and make sure that all the versions are correctly installed and all the .dll files are in the correct place. Otherwise, I am afraid I won't add much other than just googling for this error and trying to find the solution.

[Enricocloud]

Hi, just to undestand if was a compatibility problem, I'm now using the CPU tensoflow, i got a top-down-id model, PredictPoseIdentities on bonsai, but i still have an error I used the .pb file exported by bonsai, and as a Traininconfig the training_config.json of the multiclass model. How can I solve this problem?

[bruno-f-cruz]

I am not sure I follow. Bonsai is not able of exporting any .pb file...

[Enricocloud]

no sorry, I exported with selap-export, now I want to use the model that I exported to live-track the animals, but it gives me that error. How can I solve it?

[MForsthofer]

Hey, just wanted to check in if you ever figured out how to solve this? I' running into a similar issue using the PredictPoses node using a top-down model i trained in the GUI. The workflow works nicely with single instance models, but for top-down i get the same 'Object reference not set to an isntance of an object' error.

Also posted on the Sleap discussions about it here https://github.com/talmolab/sleap/discussions/1626

My workflow for testing it is just this

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.7.3"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:cv="clr-namespace:Bonsai.Vision;assembly=Bonsai.Vision"
                 xmlns:sleap="clr-namespace:Bonsai.Sleap;assembly=Bonsai.Sleap"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:FileCapture">
          <cv:FileName>D:\data\training_vids\St46\tadpole_2023-12-05T11_48_42_9.avi</cv:FileName>
          <cv:PlaybackRate>0</cv:PlaybackRate>
          <cv:StartPosition>0.5</cv:StartPosition>
          <cv:PositionUnits>Frames</cv:PositionUnits>
          <cv:Loop>true</cv:Loop>
          <cv:Playing>true</cv:Playing>
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="sleap:PredictPoses">
          <sleap:ModelFileName>C:\Users\Michael Forsthofer\OneDrive - University of Sussex\Baden Lab\Behavior tracking\TadpoleSwimTracking\SLEAP\models\231206_175613.centered_instance.n=237\frozen_graph.pb</sleap:ModelFileName>
          <sleap:TrainingConfig>C:\Users\Michael Forsthofer\OneDrive - University of Sussex\Baden Lab\Behavior tracking\TadpoleSwimTracking\SLEAP\models\231206_175613.centered_instance.n=237\training_config.json</sleap:TrainingConfig>
          <sleap:CentroidMinConfidence>0.476744175</sleap:CentroidMinConfidence>
          <sleap:PartMinConfidence>0.1744186</sleap:PartMinConfidence>
          <sleap:ScaleFactor xsi:nil="true" />
          <sleap:ColorConversion xsi:nil="true" />
        </Combinator>
      </Expression>
    </Nodes>
    <Edges>
      <Edge From="0" To="1" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>

[talmo]

Hey guys, just tagging https://github.com/talmolab/sleap/issues/1542 and @ssrinath22 who's working on this issue on the SLEAP side.