sgoldenlab / simba

SimBA (Simple Behavioral Analysis), a pipeline and GUI for developing supervised behavioral classifiers
https://simba-uw-tf-dev.readthedocs.io/
GNU General Public License v3.0
273 stars 137 forks source link

Problem in training the model #267

Closed BarakMalul closed 1 year ago

BarakMalul commented 1 year ago

Describe the bug after been through all the step to traing the model without getting any errors when I'm getting this error message:

(simba-test) C:\Users\Barak Malul>simba C:\ProgramData\anaconda3\envs\simba-test\lib\site-packages\sklearn\utils\deprecation.py:144: FutureWarning: The sklearn.metrics.classification module is deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.metrics. Anything that cannot be imported from sklearn.metrics is now part of the private API. warnings.warn(message, FutureWarning) SIMBA ERROR: Field name None could not be found in file C:\Users\Barak Malul\Documents\firstry\project_folder\csv\targets_inserted\Converted20220309_1340-1356_Cam05.csv error Exception in thread Thread-1: Traceback (most recent call last): File "C:\ProgramData\anaconda3\envs\simba-test\lib\threading.py", line 916, in _bootstrap_inner self.run() File "C:\ProgramData\anaconda3\envs\simba-test\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "C:\ProgramData\anaconda3\envs\simba-test\lib\concurrent\futures\process.py", line 272, in _queue_management_worker result_item = reader.recv() File "C:\ProgramData\anaconda3\envs\simba-test\lib\multiprocessing\connection.py", line 251, in recv return _ForkingPickler.loads(buf.getbuffer()) TypeError: init() missing 2 required positional arguments: 'column_name' and 'file_name'

To Reproduce Steps to reproduce the behavior:

  1. Go to Train machine model
  2. Click on Train single error

Expected behavior success in training the model

Desktop (please complete the following information):

Additional context All of the values of column of desired Classifier in the csv file in the targets_inserted csv file are 0 from some reason despite there are mentioning of the specific behaviour in the BORIS imported file. the warnings in the impoting of the annotations are: SIMBA WARNING: SIMBA THIRD-PARTY ANNOTATION WARNING: Annotations file for video Converted20220309_1340-1356_Cam05 has annotations for the following behaviors ['Allogrooming', 'Wrestling/ Rough play', 'Start scoring', 'Pinning', 'Interaction', 'Grooming', 'End scoring', 'Biting'] that are NOT classifiers named in the SimBA project. SimBA will OMIT appending the data for these 8 classifiers. ❗️ SIMBA WARNING: SIMBA THIRD-PARTY ANNOTATION WARNING: SimBA found THIRD-PARTY annotations for behavior Anogenital sniffing in video Converted20220309_1340-1356_Cam05 that are annotated to occur at times which is not present in the video data you imported into SIMBA. The video you imported to SimBA has 24687 frames. However, in BORIS, you have annotated Anogenital sniffing to happen at frame number P. These ambiguous annotations occur in 6 different frames for video Converted20220309_1340-1356_Cam05 that SimBA will remove by default. Please make sure you imported the same video as you annotated in BORIS into SimBA and the video is registered with the correct frame rate. SimBA will only append annotations made to the frames present in the pose estimation data. ❗

sronilsson commented 1 year ago

Hi @BarakMalul!

The error at the top: SIMBA ERROR: Field name None could not be found in file C:\Users\Barak Malul\Documents\firstry\project_folder\csv\targets_inserted\Converted20220309_1340-1356_Cam05.csv

This happens when SimBA reads in all your CSV files in the project_folder/csv/targets_inserted directory. When it does this, a check is being run to make sure the annotation for the behavior are present in each file. In this case the classifier appears to be named None. SimBA checks for a column with the header None, it can't find it in file Converted20220309_1340-1356_Cam05, and that's then it throws you this error.

I suspect the issue is that the classifier isn't called None, and that SimBA therefore is looking for the wrong column. If you are training a single classifier, None is the default value you haven't set the hyperparameters.

Two options to fix, (i) In your project_config.ini file, navigate to the [create ensemble setting][classifier] entry and change it manually if it says None to the name of your classifier:

image

Or (ii) The entry should be set in the GUI when you click SAVE SETTINGS (GLOBAL ENVIRONMENT) in the SETTING menu

image

If you missed this, [create ensemble setting][classifier] will read None as it doesn't know which model it should be training.

BarakMalul commented 1 year ago

Thank you! And do you have a way to fix the problem I discussed above in the "Additional context"?

sronilsson commented 1 year ago

Is the name of the classifier Anogenital sniffing? Can you share your BORIS file for video Converted20220309_1340-1356_Cam05 here and I can take a look? SimBA finds a frame number called P, which is not a number. Not sure how that could happen and would be good to see the file.

BarakMalul commented 1 year ago

The name of the classifier is Anogenital sniffing. the boris file which is imported is attached Converted20220309_1340-1356_Cam05.csv I would like to mention that I followed the steps from the tutorial and the csv created from the boris file is in another fomat so i made him fit to the example in the tutorial. this is the original export csv from the BORIS file without changes: Converted20220207_0000-2359_Cam02 Pre CM, P1 (102.1 & 102.2).csv

sronilsson commented 1 year ago

Thanks @BarakMalul, I will try with the Converted20220309_1340-1356_Cam05.csv and see how it goes:

NOTE, since a couple of weeks, you should work if your BORIS files are either in THIS or THIS format.

sronilsson commented 1 year ago

Hi @BarakMalul,

It seems like all your annotated anogenital sniffing events are POINT events. E.g., they do not have a start and an end-time, from the documentation:

image

In your BORIS file though, all the anogenital sniffing events appears to be POINT events, without starts and stops:

image

I will insert better error msg so this is clearer.

BarakMalul commented 1 year ago

It's worked! Thank you so much! is there a simple way to take into the training of the model the point which is not make a difference in time between every two point and lable "start" and "point"?

BarakMalul commented 1 year ago

and another issue if it's possible - this error poped when start training the model:

File "C:\ProgramData\anaconda3\envs\simba-test\lib\tkinter__init.py", line 1705, in call__ return self.func(*args) File "C:\ProgramData\anaconda3\envs\simba-test\lib\site-packages\simba\SimBA.py", line 361, in button_trainmachinemodel = Button(label_trainmachinemodel,text='TRAIN SINGLE MODEL (GLOBAL ENVIRONMENT)',fg='blue',command = lambda: threading.Thread(target=self.train_single_model(config_path=self.config_path)).start()) File "C:\ProgramData\anaconda3\envs\simba-test\lib\site-packages\simba\SimBA.py", line 577, in train_single_model model_trainer.save_model() File "C:\ProgramData\anaconda3\envs\simba-test\lib\site-packages\simba\model\train_rf.py", line 224, in save_model self.save_rf_model(self.rf_clf, self.clf_name, self.model_dir_out) File "C:\ProgramData\anaconda3\envs\simba-test\lib\site-packages\simba\mixins\train_model_mixin.py", line 707, in save_rf_model pickle.dump(rf_clf, open(save_path, 'wb')) FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\Barak Malul\Documents\firstry\project_folder\models\generated_models\Wrestling/ Rough play.sav'

sronilsson commented 1 year ago

Ah is the classifier called Wrestling/ Rough play ? If so, I think SimBA is having trouble with the final '/' in the path, it is trying to save the classifier named Rough play in the directory Wrestling. That directory doesn't exist as it is part of the classifier name.

sronilsson commented 1 year ago
It's worked! Thank you so much!
is there a simple way to take into the training of the model the point which is not make a difference in time between every two point and lable "start" and "point"

I'm not sure I understand complete: do you want to convert every other anogenital sniffing POINT to START and STOP? I don't have any code for that in SimBA...

BarakMalul commented 1 year ago

Ah is the classifier called Wrestling/ Rough play ?

Yes, I changed the classifier to Wrestling/ Rough play because it is describe behaviour that have start and end point in contrast to anogenital sniffing.

BarakMalul commented 1 year ago
It's worked! Thank you so much!
is there a simple way to take into the training of the model the point which is not make a difference in time between every two point and lable "start" and "point"

I'm not sure I understand complete: do you want to convert every other anogenital sniffing POINT to START and STOP? I don't have any code for that in SimBA...

Yes, ok I'll do it manually :)

sronilsson commented 1 year ago

Sorry, updated the answer above while you where typing:

I think SimBA is having trouble with the final '/' in the path, it is trying to save the classifier named Rough play in the directory Wrestling. That directory doesn't exist as it is part of the classifier name

I will insert a warning about that too

BarakMalul commented 1 year ago

It seems like it indeed was the problem. thank you so much for the detailed and supportive assistance. appriciate it!