search

talmolab / sleap

A deep learning framework for multi-animal pose tracking.
https://sleap.ai
Other
435 stars 97 forks source link

Issues in training and inference data #1661

Closed lenacochrane closed 10 months ago

lenacochrane commented 10 months ago

Bug description

So I am new to SLEAP and am really enjoying the interface : ) I want to use it to track larvae in a petri dish. I have had no issue in training (to my knowledge). I am able to upload a video and then run tracking with it generating a model. However, when I ran inference it came up with an error message. After trying several times I examined my terminal and this error message was repeatedly generated:

loc("mps_select"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/0032d1ee-80fd-11ee-8227-6aecfccc70fe/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":294:0)): error: 'anec.gain_offset_control' op result #0 must be 4D/5D memref of 16-bit float or 8-bit signed integer or 8-bit unsigned integer values, but got 'memref<2x3x1x1xi1>' Versions:

I then checked any error messages which may have occured during my tracking and terminal repeated such:

loc("mps_resize"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/0032d1ee-80fd-11ee-8227-6aecfccc70fe/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":180:0)): error: 'anec.resize' op Invalid configuration for the following reasons: Error: Invalid tensor format: "fp32”

Further, when the error message was displayed following running Inference, I could see that the animals had predicted tracking marks on them! Image:

Screenshot 2024-01-12 at 17 12 20

To add to the confusion, my colleague is using macOS Sonoma Version 14.2.1, with SLEAP also located in a miniconda base, and it is seemingly okay.

I am unsure how to sort this and I hope someone can help me - would really like to start using SLEAP properly soon! : )

Expected behaviour

Assessing the Inference data.

Actual behaviour

Error message occurs.

Your personal set up

I installed SLEAP using the following command lines:

conda update -n base conda conda install -n base conda-libmamba-solver conda config --set solver libmamba

conda create -y -n sleap -c conda-forge -c anaconda -c sleap sleap=1.3.3

conda activate sleap

sleap-label

SLEAP is located in my base : /Users/cochral/miniconda3

I am using a MacBook Pro; Chip: Apple M1 Pro; macOS Sonoma Version 14.2.1

Environment packages ``` # paste output of `pip freeze` or `conda list` here ```
Logs ``` # paste relevant logs here, if any ```

Screenshots

How to reproduce

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error
roomrys commented 10 months ago

Hi @lenacochrane,

I still need to research the message you received in the terminal. Thanks for that - it is also helpful to us to see a full traceback of where the message occurred, so copy/pasting your terminal from Happy SLEAPing is always nice. Do you mind providing that for us if possible?

For a bit of researching it looks like the message is accompanied by slowdown during inference - is that what you experienced? When you say the error message pops up, do you mean a dialog popped up with an error (any screenshots of that)? To be clear, inference ran and predictions were generated? I might need a bit more help understanding the symptoms of your problem - thanks for being patient!

Thanks in advance for the clarifications, Liezl

Error message research (ongoing)

lenacochrane commented 10 months ago

Hi Liezl,

Thank you for your response. I will send the terminal messages this weekend as I don’t have them saved. The error message I am referring to is the pop up where it says inference ran on frames but no predictions found or something (the error pop up was attatched in the image). The error message changed on whether I did current frame or random 20 frames on inference. Also as an update, weirdly my colleague just uploaded my tracking files into his SLEAP and then ran the inference and it worked perfectly. So the files must be okay but not comparable with my computer??? His is also Mac and he is also using miniconda ! So very strange?

Many thanks,

Lena

Sent from Outlook for iOShttps://aka.ms/o0ukef

From: Liezl Maree @.> Sent: Friday, January 12, 2024 7:26:26 PM To: talmolab/sleap @.> Cc: lenacochrane @.>; Mention @.> Subject: Re: [talmolab/sleap] Issues in training and inference data (Issue #1661)

Hi @lenacochranehttps://github.com/lenacochrane,

I still need to research the message you received in the terminal. Thanks for that - it is also helpful to us to see a full traceback of where the message occurred, so copy/pasting your terminal from Happy SLEAPing is always nice.

It seems totally possible that it is just a warning as it seems that inference completed successfully. When you say the error message pops up, do you mean a dialog popped up with an error (any screenshots of that)?

Thanks in advance for the clarifications, Liezl

— Reply to this email directly, view it on GitHubhttps://github.com/talmolab/sleap/issues/1661#issuecomment-1889834991, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BDSDFNZD2TYWJN6GEFGUCN3YOGE6FAVCNFSM6AAAAABBYRX5EGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBZHAZTIOJZGE. You are receiving this because you were mentioned.Message ID: @.***>

lenacochrane commented 10 months ago

Hello, I hope all is well. I just wanted to say that today I uninstalled the program and reinstalled it and now it is working perfectly! I am unsure what happened but thank you very much for your help!

Many thanks, Lena

roomrys commented 10 months ago

Thanks for the update Lena. I will close this for now, but if you or anyone else runs into the issue, please comment below to help us track it down.

Thanks, Liezl