Data import issues

[kapilsaxena33]

Hi All, Thanks for help, while in the active learning tab found this error, any idea how to solve it? Please

[JensBlack]

Sorry, but I need much more info than that. Can you post the config.ini, a screenshot or copy-paste of the complete error (console preferred and some general info on what you were doing when the issue occurred? Thank you!

[kapilsaxena33]

HI Jens, What i did is uploaded the DLC files and BORIS files as input and set the frame rate at 15 FPS, run the uplad data, it created teh config file, that i used to extract features. There i got the error as well.

ValueError: Found input variables with inconsistent numbers of samples: [319, 318] IndexError: list index out of range

So I manipulated the bin size
And minimum duration of behavior transitions vary it until the error got resolved, i dont understand how this works. For me a value of 0.6 to 0.55 solved the errors.

Then i went to the active learning tab In active learning In INITIAL SAMPLING RATIO Lover value (0.01-0.03) resulted in ValueError: need at least one array to concatenate

Increasing INITIAL SAMPLING RATIO to 0.04 resulted in ValueError: operands could not be broadcast together with shapes (5,) (4,) or Initial sample ratio ValueError: operands could not be broadcast together with shapes (7,) (8,)

Also attaching the required stuff. Config file as txt doc. and error screen shot. config.ini.txt Dump the consol in next message.

thanks for helping, if you are available, we can do a zoom on your preferred time and i can show you what is happening. Or i can also send you my files and you can run on your system to see if the input files have any issue, which ever works to solve the problem. Best kapil

[kapilsaxena33]

consol dump.txt

consol dump

[JensBlack]

Ok, this looks like a propagation issue from the initial data import. Please verify for me that:

• the pose estimation data was done on a video with the specified framerate (15)
• the annotation data is aligned to the correct pose estimation file (you can check this in the config file)
• the annotation files are for the entire length of the pose estimation file
• the annotation file is exported to 10 Hz (0.1 sec bins) or directly to 15 Hz (like the pose estimation)

It looks like during data import it was not able to match pose data with annotations, which results in a length mismatch down the line. Interesting though, that you were able to continue.

So I manipulated the bin size And minimum duration of behavior transitions vary it until the error got resolved, i dont understand how this works. For me a value of 0.6 to 0.55 solved the errors.

I am not sure from your info why you were able to alter the bin sizes to "fix" this. But it is not the intended behavior.

Note, however, that you are not able to use bins that are smaller than 2 frames (which would be ca. 0.122 sec for 15 Hz). So this seems to be a separate issue coming from your unusually low framerate.

Then i went to the active learning tab In active learning In INITIAL SAMPLING RATIO Lover value (0.01-0.03) resulted in ValueError: need at least one array to concatenate

From the screenshot, I see that you are using a initial sampling ratio that would give you 0 samples for at least one class, which will result in an error when running the classifier. For it to run at all in the intended way, you need at least 1 sample per class.

I suggest the following:

1. Verify the above info
2. Test this with a single pose estimation and single annotation file. Is the error still happening?
3. Increase the bin size for feature extraction to a point way above your framerate (just for testing now)
4. Choose a high percentage for the initial samples (just for testing now)
5. Increase the number of included data and verify if you have "corrupted" data
6. report back with the results

I am confident, we will find a solution to this. Again, thank you for being an early adopter and providing your valuable input!

[kapilsaxena33]

You are right about the mismatch between the DLC data file and the BORIS data files, primarily because DLC works on frames and the output is also frames (lines in sheet are exactly as many frames in the video), but in BORIS when you export to Binary it asked what is the minimum duration of time which we give in millisecound based on based on 15 FPS rate video rate which is 06666666667 s. but you can only give 0.66 or0.67. That result in a mismatch in the number of rows in both files, do you find a workaround to this problem?

[JensBlack]

Yes, we actually upsample to match the framerates of the video when importing the data. To avoid the issue with 0.33333333... I recommend exporting at 0.1 sec (10 Hz) and leave the pose estimation as is. A-SOiD upsamples to 30 Hz, then by itself. I'll make a note that we should clarify the process in the app. This should solve some of the issues you are experiencing. Please report back if this solved your issues completely.

[kapilsaxena33]

Cool, i love it, you solved the issue basically, it is clear. The BORIS developer is also a friend and i have asked for this Export with Frame feature, he assured that it will be included soon, lets hope for the best.

[JensBlack]

THAT would be fantastic!

Glad that I was able to solve your issue. Closing this thread.

[JensBlack]

Linking the feature request here: https://github.com/olivierfriard/BORIS/issues/544#issue-1408517745