sgoldenlab / simba

SimBA (Simple Behavioral Analysis), a pipeline and GUI for developing supervised behavioral classifiers
https://simba-uw-tf-dev.readthedocs.io/
GNU General Public License v3.0
290 stars 141 forks source link

Impossible to import h5 elipse data from DLC #210

Closed DorianBattivelli closed 2 years ago

DorianBattivelli commented 2 years ago

Hello,

Sorry for reporting this small bug but I can't import my el.h5 file, the terminal reports:

H5-error

What could be the error? FYI, I'm using 0.95.9 simba version on windows. Thank you, Best,

DorianBattivelli commented 2 years ago

I precise that it's a second importation of h5 file for the same video, since the first h5 contained some mistakes. I erased the coresponding csv file (from the first h5 file) in the csv folder.

sronilsson commented 2 years ago

Hi @DorianBattivelli thanks for reporting! Is it possible that you have some hidden files located within the directory you are importing the files from, or hidden files within your SimBA project_folder/videos directory? https://support.microsoft.com/en-us/windows/show-hidden-files-0320fe58-0117-fd59-6851-9b7f9840fdb2#:~:text=Select%20the%20Start%20button%2C%20then,drives%2C%20and%20then%20select%20OK.

DorianBattivelli commented 2 years ago

Thank you for the answer. I checked and I don't have any hidden in these directories

sronilsson commented 2 years ago

I bumped into this error myself a few times when I was troubleshooting something different. It happened when I have a file in the videos folder that does not have a file extension (or only an extension, no file-name) and I was about to insert a fix. Here, in the image, for example, there is a file called .DS_store in my videos directory that is hidden which is causing the code to fail (it can't find the video's name, only extension). I will insert a fix for this now, and let's see if it fixes the issue on your end, if not we can dig more.

image

sronilsson commented 2 years ago

@DorianBattivelli - if you update to 0.97.4 - do you still see the error?

DorianBattivelli commented 2 years ago

Thank you, unfortunatly the update did not fix the bug. Here the files and corresponding paths I use to import the h5 file: Schermata 2022-09-06 alle 16 28 16

sronilsson commented 2 years ago

Thanks @DorianBattivelli - I cant see the entire file explorer window. To troubleshoot, could you also tick the File name extensions box next to the Hidden items box and send me the screengrab?

If you move the Bh5 folder to a different location, does the error persist?

DorianBattivelli commented 2 years ago

I also tried by moving the folder on the desktop, and the problem persists. Schermata 2022-09-06 alle 16 42 06

I precise that I do not import the video since it is already present in the video folder. Could that create the error?

sronilsson commented 2 years ago

Yes, that should not cause an issue. In version 0.97.4, I inserted a print statement, that shows all the files SimBA finds in the video folder, before it starts to scan which one belongs to your el.h5 file.

Did you see that print statement?

DorianBattivelli commented 2 years ago

I also tried by moving the folder on the desktop, and the problem persists. Schermata 2022-09-06 alle 16 42 06

I precise that I do not import the video since it is already present in the video folder. Could that create the error?

Yes, that should not cause an issue. In version 0.97.4, I inserted a print statement, that shows all the files SimBA finds in the video folder, before it starts to scan which one belongs to your el.h5 file.

Did you see that print statement?

Not sure to understand, I should find the statement in the error that returns the terminal?

sronilsson commented 2 years ago

Hi - sorry I should have been clearer (I also put the print statement in the wrong place). First, update to simba 0.97.5 with pip install simba-uw-tf-dev --upgrade.

Then, when you import your data, SimBA prints out all of the files that are found inside your project_folder/videos directory, as is marked in the red box in the screengrab below. In your case, one of these files listed, does not have a file ending, or a filename. SimBA tries to identify the video file format, and the file-name, and it fails. This would happen for example if one if the files inside your project_folder/videos directory begins with the character .

Untitled 3

DorianBattivelli commented 2 years ago

After updating, here what I get: Schermata 2022-09-06 alle 17 50 04

sronilsson commented 2 years ago

@DorianBattivelli - I've inserted some code that should filter out all subdirectories and hidden files, and show you the name of the offending file if you still hit an error. Can you update SimBA to 0.97.5 and let me know how it goes?

DorianBattivelli commented 2 years ago

Amazing, it solved the issue, thank you very much!

DorianBattivelli commented 2 years ago

Well actually I am stuck at the next tab, when trying to set the video parameters: Schermata 2022-09-07 alle 00 20 23

sronilsson commented 2 years ago

Alright - hang on, I will insert a more solid fix. There is likely to be some hidden file in that folder.

sronilsson commented 2 years ago

I'm sorry I don't have a fix. But hopefully I have an update that can help us figure out what is going on. If you update to version 0.97.7, and repeat the same process, what is the error message you see in the main SimBA terminal?

What you should see something like this - SIMBA ERROR: ThisFilename is not a valid filepath

DorianBattivelli commented 2 years ago

Thank you for the update, Indeed the error message is as you said

Loading F:/Territorial_Arena/Paper/Protocol_1/CD1/Simba/B1-MS_US/project_folder/project_config.ini... SIMBA ERROR: F:/Territorial_Arena/Paper/Protocol_1/CD1/Simba\B1-MS_US\project_folder\videos\Bh5 is not a valid filepath

DorianBattivelli commented 2 years ago

Okm annd removing the h5 folder fixed the problem ,) thank you!