Extract features - inconsistent number of samples

Hi, I am using A-SOiD and I am getting stuck on the extract features tab. I used B-SOiD with success, and am trying to use A-SOiD in conjunction with labels I have from BORIS. I originally had the timestep for BORIS labels as 0.002 s/step, which is the framerate of the captured videos and syncs up with my DeepLabCut kinematic data. But I saw in the files that A-SOiD requires >0.033 s/step. Fair enough. I'm not sure if this was necessarily a reccomendation, but I saw in load_labels_boris() that you saved BORIS labels as 100 ms time steps, so I went ahead and did that. I'm not sure if I need to downsample my DeepLabCut file to match the BORIS sampling so they are the same length? I'll probably try that in the mean time. But any help?

This is the full error message: ValueError: Found input variables with inconsistent numbers of samples: [4593, 3675] Traceback: File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\streamlit\scriptrunner\script_runner.py", line 557, in _run_script exec(code, module.dict) File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\asoid\app.py", line 332, in main() File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\asoid\app.py", line 328, in main application_function() File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\asoid\apps\B_extract_features.py", line 101, in main extractor.main() File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\extract_features.py", line 524, in main self.shuffle_data() File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\asoid\utils\extract_features.py", line 494, in shuffle_data X_train, X_test, y_train, y_test = train_test_split(self.scaled_features, self.targets_mode, File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\sklearn\model_selection_split.py", line 2559, in train_test_split arrays = indexable(arrays) File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\sklearn\utils\validation.py", line 443, in indexable check_consistent_length(result) File "C:\Users\nicko\anaconda3\envs\asoid-test\lib\site-packages\sklearn\utils\validation.py", line 397, in check_consistent_length raise ValueError(

Hi @nich999,

thanks for trying A-SOiD. Let me try to help you with this by first clarifying the intention behind the BORIS export.

EDIT: I've checked the recent release and realised that they increase the available sample rate at which you can export files. Great!

AFAIK Boris does not allow a binary export beyond 0.1 sec time steps, so the easiest solution is to upsample the data from that to match the framerate. That said, there is no reason for you to downsample your data if you have already a matching sample rate. The current version of the Boris import checks the sample rate by extracting the first time step and then calculates whether it needs to upsample the labels. If the sample rate matches the framerate you've entered, it will simply use the unchanged labels.

Second, the issue you are facing is as you already guessed, most likely a mismatch between the pose estimation and label files.

To clarify: Do you have a time step of 0.002 sec (500 Hz sample rate) for your behavioral labels? That seems very high for a supervised annotation, but fair enough.

This would mean that your video is roughly 9.2 seconds (4593 frames / 500 fps) long, right? This makes your use case very interesting. I would love to hear more about it.

Can you confirm the original size of your Boris file for me? It should have 4593 entries as well.

The mismatch suggests that this file is considerably shorter (7.3 seconds, 3675 entries). That said, this might be a mismatch in the file associations during project creation:

Can you confirm that you associated the correct order of label files, with the corresponding pose estimation files in the initial data import step?

During that step, there is no check for mismatching, so the first moment where this would lead to an error is during feature extraction.

Another alternative issue is the Boris export itself: You need to export the labels to the full video length (not the length of the annotations)

Export using binary export: grafik

Select Media file(s) duration to match the lenght of the pose estimation file: grafik

Please also post the content of your config file, for me to check for any other apparent issues. It would be great if you could send me at least the first 5 rows (incl. header) of the Boris file in question, so I can confirm that it is a suitable format.

Hi Jens, thanks for the response. The project I'm working on is analysis of kangaroo rat hopping kinematics. I was curious about how B-SOiD's unsupervised techniques would translate to behavior as it pertains to gait. So I'm interested in its application to labeling unseen kangaroo rat hopping videos wrt the gait cycle. I'm also curious about studying the behavior of the tail wrt the gait cycle, but that's a bit secondary right now. I got B-SOiD to work somewhat well, my primary issue was that it was usually a ~5 frame window for identifying key events like when the kangaroo rat lands or takes off. But it identified stance and swing very well. I'm hoping A-SOiD may help tune it a bit with the information from our BORIS labels.

I haven't really seen any applications of B-SOiD to gait in any of the papers that cited B-SOiD, so this may be a slightly atypical application. I found the A-SOiD preprint on biorxiv recently and needed to check it out. I'm optimistic about the application! It has also made me consider using streamlit for some of the code I write in our lab.

I am working with videos that are typically under a minute of real-time hopping. The videos were recorded at 500fps and show at 25fps so they appear much slower when watching them. Now that I think about it, I may need to export at the fps of the video and not the FPS it was recorded... Update: This was definitely an issue. I exported at 1/25fps and this matches the frame # of my DeepLabCut file. Yay.

Another alternative issue is the Boris export itself: You need to export the labels to the full video length (not the length of the annotations) This was definitely an issue. We just recently switched to BORIS for labeling. It is nice to know this could've been a cause for issues in the future.

These two fixes seemed to be the main issue. I was exporting the BORIS files as you stated, I just exported for the length of the observations rather than the entire video. I am now onto other issues. I'll try to see if I can fix them but if not I'll open a separate thread for it. Thank you very much!!

YttriLab / A-SOID

Extract features - inconsistent number of samples #30