No Behaviors Marked as None?

[kjmartin44]

Describe the bug After normal frame labelling in the GUI I attempt to use machine learning but I get an error for apparently having no behaviors marked as None. However, I do indeed have frames that contain no behavior labels.

To Reproduce Steps to reproduce the behavior:

1. Label Behavior>Label Behavior
2. Train Machine Model> Train Single Model

SIMBA MISSING COLUMN ERROR: Data for video 20240401SPT35_1-04012024154559 does not contain any annotations for behavior None. Delete classifier None from the SimBA project, or add annotations for behavior None to the video 20240401SPT35_1-04012024154559

Expected behavior The Machine Model Successfully begins training

Desktop (please complete the following information):

• OS: Windows 10
• Python Version 3.10
• Are you using anaconda? Yes

[sronilsson]

Hi @kjmartin44!

This suggest that when SimBA reads in your project_config.ini, to check out what the expected classifier names are, there are some missing values that SimBA doesn't know what to do with, for example. There could be something looking like this:

If you drag-and-drop the project_config.ini (you may have to zip it) into this thread I can have a look at it to see where the error could come from?

[kjmartin44]

project_config - Copy.zip Here is my config.

Thank you

[sronilsson]

Got it - thanks.

Check on line 73 in that file:

It says None. That value is typically set when you set your hyperparameters in this menu:

You can change it manually in the project_congig, or do it through the GUI and click "SAVE SETTINGS (GLOBAL ENVIRONMENT)" at the bottom of the MACHINE MODEL SETTING pop-up.

Let me know if that makes sense.

[kjmartin44]

SIMBA INVALID ML TRAINING SET ERROR: 28 feature columns exist in some files, but missing in others. The feature files are found in the project_folder/csv/targets_inserted directory. SimBA expects all files within the project_folder/csv/targets_inserted directory to have the same number of features: the first 10 column names with mismatches are: ['Movement_Nose', 'Movement_Right Hand', 'Movement_R1', 'Movement_R2', 'Movement_R3', 'Movement_R4', 'Movement_RElbow', 'Movement_RShoulder', 'Movement_Left Hand']

I am now getting this error, I only have one file that has been labelled though so it should not vary. However, I was having an issue with PIL and I had to downgrade it in order for excel file writing to be correct, but I labelled this file before doing so. should the targets_inserted.csv look almost identical to the deeplabcut inputs? because that is what it looks like for me now. 20240401SPT35_1-04012024154559.csv

[sronilsson]

Hi @kjmartin44 - how many files exist within the project_folder/csv/targets_inserted directory? Can you send me a screengrab of this directory?

I can't soenload the CSV. I think you may have to zip it before uploading it here as previously.

[kjmartin44]

Here is both the screengrab and the file 20240401SPT35_1-04012024154559.zip

[sronilsson]

There are a lot of infs in there for the first 28 features. That probably has something to do with it, but I am not sure yet why they are there instead of real values. How big is your entire project_folder? If you can share that, maybe through a gdrive link or similar, I can figure out how this has happaned?

[sronilsson]

PS. what is the FPS of your video?

[kjmartin44]

The project folder is 42 gb. I am uploading it to Drive and can email it to you when it is finished. My videos are 499 fps

[sronilsson]

that's probably where the issues lie, with the FPS...

Speculating but... To save on RAM requirements, data gets stored as 32 bit floats in SimBA. Then we want to calculate how much the animals have moved over the last N millisecond... e.g., the code calculates frame-by-frame movement and then aggregates that to the mean / sum over rolling windows. If you have fps of 499 the frame-by-frame movement will be tiny or maybe 0 as a float32 and when aggregated. Then we could get the infs.

For a human recognize the behaviors you are interested in the videos by eye, is 499 FPS a requirement, if not - do you know what the lower limit is?

[kjmartin44]

I was testing out simba to see if I could identify sub-movements within a task as I can with deepethogram. My animals move so fast that some of the movements are complete within 3 frames at 499 fps so I cannot lower this number. However, I have some body parts that do not move much while others are incredibly active. Would removing those mostly still tracked body parts from the project possibly help?

[kjmartin44]

Or would you recommend editing the videos so that the video is the same but the frame rate is 40 fps, so that the videos are analyzed in slow motion?

[sronilsson]

Ah OK, if a behavior is completed within 3 frames at 499 fps, there is no room to resample the videos., thats less than a millisecond i think very short. I'm curios about this, let me troubleshoot with the project and fix infs once I got it and get back to you.

[kjmartin44]

I removed all of the body parts that have little to no motion from the csvs and I still have the issue of the inability to read the movement columns because it marks them as 0 or inf. Have you found any solutions so that I am able to use my data?

[sronilsson]

Sorry @kjmartin44 - fell between the chairs thanks for reminding me I'm going to try to replicate your error after setting the FPS to 499

[sronilsson]

Hi @kjmartin44 - I gave it a go trying to replicate this error on some 499 FPS projects and I couldn't do it...

Which version of SimBA are you running and can see with pip show simba-uw-tf-dev ?

[kjmartin44]

Here is the info on that before the large amount of info text. Could it be due to the fact that I am using filtered data from DLC? or the fact my videos can be around 7000 frames long?

Name: Simba-UW-tf-dev Version: 2.0.7 Summary: Toolkit for computer classification and analysis of behaviors in experimental animals Home-page: https://github.com/sgoldenlab/simba Author: Simon Nilsson, Jia Jie Choong, Sophia Hwang Author-email: sronilsson@gmail.com License: GNU General Public License v3 (GPLv3) Location: c:\users\martiki\appdata\local\anaconda3\envs\simba\lib\site-packages Requires: cefpython3, dash, dash-color-picker, dash-colorscales, dash-core-components, dash-html-components, dtreeviz, eli5, ffmpeg-python, geos, graphviz, h5py, imbalanced-learn, imgaug, imutils, joblib, kaleido, matplotlib, numba, numexpr, numpy, opencv-python, pandas, Pillow, plotly, psutil, pyarrow, pyglet, pyyaml, scikit-image, scikit-learn, scipy, seaborn, shap, shapely, statsmodels, tables, tabulate, tqdm, trafaret, xgboost, xlrd, yellowbrick

and if it helps here is what comes up with conda list on my env attrs 24.2.0 pypi_0 pypi blas 1.0 mkl blinker 1.8.2 pypi_0 pypi blosc2 2.7.1 pypi_0 pypi brotli 1.1.0 pypi_0 pypi bzip2 1.0.8 h2bbff1b_6 ca-certificates 2024.7.2 haa95532_0 cairo 1.16.0 haedb8bc_5 cefpython3 66.0 pypi_0 pypi click 8.1.7 pypi_0 pypi colorama 0.4.6 pypi_0 pypi colour 0.1.5 pypi_0 pypi cudatoolkit 11.8.0 hd77b12b_0 cudnn 8.9.2.26 cuda11_0 cycler 0.12.1 pypi_0 pypi dash 1.14.0 pypi_0 pypi dash-color-picker 0.0.1 pypi_0 pypi dash-colorscales 0.0.4 pypi_0 pypi dash-core-components 1.10.2 pypi_0 pypi dash-html-components 1.0.3 pypi_0 pypi dash-renderer 1.6.0 pypi_0 pypi dash-table 4.9.0 pypi_0 pypi dav1d 1.2.1 h2bbff1b_0 dtreeviz 0.8.1 pypi_0 pypi eli5 0.10.1 pypi_0 pypi et_xmlfile 1.1.0 py310haa95532_0 expat 2.6.2 hd77b12b_0 ffmpeg 6.1.1 h9eb6184_0 ffmpeg-python 0.2.0 pypi_0 pypi flask 3.0.3 pypi_0 pypi flask-compress 1.15 pypi_0 pypi fontconfig 2.14.1 h9c4af85_2 fonttools 4.53.1 pypi_0 pypi freetype 2.12.1 ha860e81_0 future 1.0.0 pypi_0 pypi geos 0.2.3 pypi_0 pypi glib 2.78.4 hd77b12b_0 glib-tools 2.78.4 hd77b12b_0 graphite2 1.3.14 hd77b12b_1 h5py 3.11.0 pypi_0 pypi harfbuzz 4.3.0 hb646838_2 icu 73.1 h6c2663c_0 imageio 2.35.0 pypi_0 pypi imbalanced-learn 0.12.2 pypi_0 pypi imgaug 0.4.0 pypi_0 pypi imutils 0.5.2 pypi_0 pypi intel-openmp 2023.1.0 h59b6b97_46320 itsdangerous 2.2.0 pypi_0 pypi jinja2 3.1.4 pypi_0 pypi joblib 1.4.2 pypi_0 pypi kaleido 0.2.1 pypi_0 pypi kiwisolver 1.4.5 pypi_0 pypi lazy-loader 0.4 pypi_0 pypi libffi 3.4.4 hd77b12b_1 libglib 2.78.4 ha17d25a_0 libiconv 1.16 h2bbff1b_3 libogg 1.3.5 h2bbff1b_1 libpng 1.6.39 h8cc25b3_0 libtheora 1.1.1 h8cc25b3_3 libxml2 2.10.4 h0ad7f3c_2 llvmlite 0.42.0 pypi_0 pypi lxml 5.3.0 pypi_0 pypi markupsafe 2.1.5 pypi_0 pypi matplotlib 3.5.0 pypi_0 pypi mkl 2023.1.0 h6b88ed4_46358 mkl-service 2.4.0 py310h2bbff1b_1 mkl_fft 1.3.8 py310h2bbff1b_0 mkl_random 1.2.4 py310h59b6b97_0 msgpack 1.0.8 pypi_0 pypi ndindex 1.8 pypi_0 pypi networkx 3.3 pypi_0 pypi numba 0.59.1 pypi_0 pypi numexpr 2.10.0 pypi_0 pypi numpy 1.26.4 py310h055cbcc_0 numpy-base 1.26.4 py310h65a83cf_0 opencv-python 4.8.1.78 pypi_0 pypi openpyxl 3.1.5 py310h827c3e9_0 openssl 3.0.14 h827c3e9_0 packaging 24.1 pypi_0 pypi pandas 2.2.2 pypi_0 pypi patsy 0.5.6 pypi_0 pypi pcre2 10.42 h0ff8eda_1 pillow 9.5.0 pypi_0 pypi pip 24.2 py310haa95532_0 pixman 0.40.0 h2bbff1b_1 plotly 4.9.0 pypi_0 pypi psutil 6.0.0 pypi_0 pypi py-cpuinfo 9.0.0 pypi_0 pypi pyarrow 6.0.1 pypi_0 pypi pyglet 2.0.10 pypi_0 pypi pyparsing 3.1.2 pypi_0 pypi python 3.10.14 he1021f5_1 python-dateutil 2.9.0.post0 pypi_0 pypi python-graphviz 0.11 pypi_0 pypi pytz 2024.1 pypi_0 pypi pyyaml 5.3.1 pypi_0 pypi retrying 1.3.4 pypi_0 pypi scikit-image 0.23.2 pypi_0 pypi scikit-learn 1.4.2 pypi_0 pypi scipy 1.13.0 pypi_0 pypi seaborn 0.11.0 pypi_0 pypi setuptools 72.1.0 py310haa95532_0 setuptools-scm 8.1.0 pypi_0 pypi shap 0.35.0 pypi_0 pypi shapely 2.0.2 pypi_0 pypi simba-uw-tf-dev 2.0.7 pypi_0 pypi six 1.16.0 pypi_0 pypi sqlite 3.45.3 h2bbff1b_0 statsmodels 0.14.2 pypi_0 pypi tables 3.9.2 pypi_0 pypi tabulate 0.9.0 pypi_0 pypi tbb 2021.8.0 h59b6b97_0 threadpoolctl 3.5.0 pypi_0 pypi tifffile 2024.8.10 pypi_0 pypi tk 8.6.14 h0416ee5_0 tomli 2.0.1 pypi_0 pypi tqdm 4.30.0 pypi_0 pypi trafaret 2.1.1 pypi_0 pypi tzdata 2024.1 pypi_0 pypi vc 14.40 h2eaa2aa_0 vs2015_runtime 14.40.33807 h98bb1dd_0 werkzeug 3.0.3 pypi_0 pypi wheel 0.43.0 py310haa95532_0 xgboost 0.90 pypi_0 pypi xlrd 1.2.0 pypi_0 pypi xz 5.4.6 h8cc25b3_1 yellowbrick 1.5 pypi_0 pypi zlib 1.2.13 h8cc25b3_1 zstandard 0.23.0 pypi_0 pypi

[sronilsson]

It is unlikely to be the 7k frames, it's also unlikely the SimBA version. I haven't updated these functions for a while.

It may have something to do with the missing values. But only if there are tons would it explain the infs.

Can you share with me your project_folder/csv/outlier_corrected_movement_location20240401SPT35_1-04012024154559.csv file? I can check how it looks.

[kjmartin44]

I had not used outlier correction before but I can try now. I had opted to skip because its only one animal. 20240401SPT35_1-04012024154559.zip

[sronilsson]

No that file looks fine, I don't see any overwhelming amounts of missing values.

I one last thing, could you share your project_folder\logs\measures\pose_configs\bp_names\project_bp_names.csv file? Maybe there is some issues with the headers files.

[kjmartin44]

project_bp_names.zip here that is

[sronilsson]

OK, one more thing :) What is the pixels per millimeter conversion factor for the video in the project_folder/logs/video_info.csv file?

[kjmartin44]

I had autopopulated the video parameters with 120 mm but i hadnt set a pixel conversion factor. Do i need to redo everything with the pixel conversion facor in place?

[sronilsson]

Ah maybe that could be it, let me see if I get an error like that on my data if the pixels per millimeter conversion factor is missing..

[sronilsson]

See how it runs after you make sure this column has a value on your end:

I will insert an error msg in the code if its not a valid integer or float value.

[kjmartin44]

Do I need to re-run other steps? After fixing that it still is giving me the same error.

[sronilsson]

If you click Skip outlier correction again and then run feature extraction, do you still see the infs?

[kjmartin44]

Yes the infs are still there

[sronilsson]

So odd - what is the pixels per millimeter?

[kjmartin44]

1.15302. I chose based on what I assumed was a mm in the video but I could be wrong there.

[kjmartin44]

would it be best for me to share my whole project folder with you over google drive?

[sronilsson]

Thank @kjmartin44 - yes I am having troubles reproducing the infs. I have used a dummy project with what I believe is your project settings (single animal, user-defined pose configuration, 499 FPS, 1.15302 pixels per millimeter) without luck. There maybe something in the data that I overlook.

[kjmartin44]

What email would work best to send? I do not have enough space on my google drive but I can share it over box.

[sronilsson]

I think that works, you can share it with sronilsson@gmail.com

[sronilsson]

Hi @kjmartin44 thanks for the data looks like a super interesting project. Two things:

i) The project is so big because the frames are extracted for every video. There is no need to do that for any process in SimBA so you could skip that.

ii) I look at the files inside the Simba_SPT2\project_folder\csv\features_extracted directory and I do not see any infs or missing values. Can you confirm that this is the case on your end too? Although I can see the infs in the targets inserted directory.

[kjmartin44]

Thank you. This is also the case for me, i assume that the infs in the targets inserted are what is causing the issue. Would copy pasting over the movement values solve my issue?

[sronilsson]

Yeah, just to check: If you try to annotate a second video, just a couple of frames here and there, and save those annotations, do you see the infs for the second video too targets inserted directory?

[sronilsson]

... would just be halpful to know, if there is something in the code related to the annotations parts that somehow introduced those infs.

[kjmartin44]

Im not sure what introduced the infs but reopening and closing labelling seems to have fixed everything. However my machine learning will not run because of an attribute error. self.save_rf_model(self.rf_clf, self.clf_name, self.model_dir_out) AttributeError: 'TrainRandomForestClassifier' object has no attribute 'rf_clf'

[sronilsson]

Do you have a full copy paste of the error msg?

[kjmartin44]

Exception in Tkinter callback Traceback (most recent call last): File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\tkinter__init.py", line 1921, in call return self.func(*args) File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\site-packages\simba\ui\tkinter_functions.py", line 436, in execute_command cmd(**evaluated_kwargs) File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\site-packages\simba\SimBA.py", line 618, in train_single_model model_trainer.save() File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\site-packages\simba\model\train_rf.py", line 386, in save self.save_rf_model(self.rf_clf, self.clf_name, self.model_dir_out) AttributeError: 'TrainRandomForestClassifier' object has no attribute 'rf_clf' Exception in Tkinter callback Traceback (most recent call last): File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\tkinter\init.py", line 1921, in call__ return self.func(*args) File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\site-packages\simba\ui\tkinter_functions.py", line 436, in execute_command cmd(**evaluated_kwargs) File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\site-packages\simba\SimBA.py", line 618, in train_single_model model_trainer.save() File "C:\Users\martiki\AppData\Local\anaconda3\envs\simba\lib\site-packages\simba\model\train_rf.py", line 386, in save self.save_rf_model(self.rf_clf, self.clf_name, self.model_dir_out) AttributeError: 'TrainRandomForestClassifier' object has no attribute 'rf_clf'

[sronilsson]

What do you see in the SimBA main terminal window GUI before this happens?

What version of simba do you have installed pip show simba-uw-tf-dev ?

[kjmartin44]

Reading in 2 annotated files... Reading complete 20240229SPT1_4-02292024154616 (elapsed time: 0.1385s)... Reading complete 20240402SPT35_5-04022024155547 (elapsed time: 0.1455s)... Number of features in dataset: 34 Number of RPull frames in dataset: 1598.0 (15.079999999999998%) Training and evaluating model... Reading in 2 annotated files... Reading complete 20240402SPT35_5-04022024155547 (elapsed time: 0.1406s)... Reading complete 20240229SPT1_4-02292024154616 (elapsed time: 0.1356s)... Number of features in dataset: 34 Number of RPull frames in dataset: 1598.0 (15.079999999999998%) Training and evaluating model... and i have 2.0.7

[sronilsson]

What did you choose in this dropdown?

[sronilsson]

If not RF, could you set it to RF and try again?

[kjmartin44]

It is on RF

[sronilsson]

Can you send me the project config again and I will take a look? You may have to zip it before you drop it in this thread.

[kjmartin44]

project_config.zip Here it is

[sronilsson]

I see nothing odd.. but I'm on my phone and I don't have a computer to test it with. I was caught by hurricane on Friday and no electricity. I will try as soon as I can.