ParseError during Project Config

[carson2stoker]

Describe the bug Hi, sorry I'm to bring up another issue for you, I'm just having trouble getting this to work. I have simba installed, and I was able to create the user defined pose config fine. However, once I fill out the rest of the project config info and click generate project config which triggers an error on the command line. I don't think the files are being generated correctly because if I try to move on and import videos and tracking, I get further errors.

pandas.errors.ParserError: Error tokenizing data. C error: Expected 16 fields in line 14, saw 21

To Reproduce Steps to reproduce the behavior:

1. Open simba
2. Click on 'create a new project '
3. add user defined pose with 1 animal and 21 bp
4. add project path and name
5. add number of classifiers and name then
6. click generate project config
7. see error

Expected behavior project config files are generated correctly

Desktop (please complete the following information): OS: [e.g. iOS] Rocky Linux 8.5 (Green Obsidian) Python Version [e.g. 3.6.0] 3.6.10 Are you using anaconda? yes

Additional context I'm using FastX3 to access a cluster where I'm running simba. No messages were created in the gui. The full terminal output from the command line is listed below.

Terminal Output

``` (base) [u1208563@notchpeak1:~]$ salloc --time=4:00:00 --account=notchpeak-gpu --partition=notchpeak-gpu --nodes=1 --ntasks=1 --mem=60G --gres=gpu:2080ti:1 salloc: Pending job allocation 5167955 salloc: job 5167955 queued and waiting for resources salloc: job 5167955 has been allocated resources salloc: Granted job allocation 5167955 salloc: Waiting for resource configuration salloc: Nodes notch271 are ready for job (base) [u1208563@notch271:~]$ conda activate simba (simba) [u1208563@notch271:~]$ simba Exception in Tkinter callback Traceback (most recent call last): File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/tkinter/__init__.py", line 1705, in __call__ return self.func(*args) File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/site-packages/simba/SimBA.py", line 3013, in make_projectini file_type=self.csvORparquet.getChoices()) File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/site-packages/simba/project_config_creator.py", line 30, in __init__ self.create_configparser_config() File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/site-packages/simba/project_config_creator.py", line 174, in create_configparser_config bp_df = pd.read_csv(bp_dir_path, header=None) File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/site-packages/pandas/io/parsers.py", line 685, in parser_f return _read(filepath_or_buffer, kwds) File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/site-packages/pandas/io/parsers.py", line 463, in _read data = parser.read(nrows) File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/site-packages/pandas/io/parsers.py", line 1154, in read ret = self._engine.read(nrows) File "/uufs/chpc.utah.edu/common/home/u1208563/miniconda3/envs/simba/lib/python3.6/site-packages/pandas/io/parsers.py", line 2059, in read data = self._reader.read(nrows) File "pandas/_libs/parsers.pyx", line 881, in pandas._libs.parsers.TextReader.read File "pandas/_libs/parsers.pyx", line 896, in pandas._libs.parsers.TextReader._read_low_memory File "pandas/_libs/parsers.pyx", line 950, in pandas._libs.parsers.TextReader._read_rows File "pandas/_libs/parsers.pyx", line 937, in pandas._libs.parsers.TextReader._tokenize_rows File "pandas/_libs/parsers.pyx", line 2132, in pandas._libs.parsers.raise_parser_error pandas.errors.ParserError: Error tokenizing data. C error: Expected 16 fields in line 14, saw 21 ```

Thanks for the help!

[sronilsson]

Hi @carson2stoker! The error is kind of related to the initial error you saw. What it looks like, is that SimBA is trying to read the pose_config_names.csv located in the SimBA package which is a CSV file. However, it fails because the delimiters is not comma, or it has been corrupted in some way. When trying to fix your prior error, did you manipulate the pose_config_names.csv in some way outside of SimBA?

[carson2stoker]

No, I didn't touch the pose_config_names.csv file at all. I also tried creating a new project and resetting the user defined pose, and recreating it, but I always got the same error.

[sronilsson]

@carson2stoker thanks, I just had another user reporting the same issue on gitter, so I will see if there are any issues in code

[sronilsson]

@carson2stoker could you do me a favour and upgrade simba with pip install simba-uw-tf-dev --upgrade and let me know if you still see the error? I'm having trouble re-creating the error, but re-wrote the script in hope that it is more robust now.

[carson2stoker]

Yes, that worked! Thank you. However, another error immediately cropped up as always the case. I imported my video, that went fine. Then, I tried to upload my tracking data generated in sleap. I did pick the .slp file type. It gave me the following error.

FAILED: SimBA could not find a video file resenting test2.6 in the project_folder/videos directory

I didn't see anything in the issues page about this and tried replacing the video path in the sleap files but that didn't work either.

Edit: to clarify, that is the message that displayed in the gui. The terminal gave me this error.

TypeError: expected str, bytes or os.PathLike object, not NoneType

[sronilsson]

Hi @carson2stoker - one step forward!! To clarify, have you imported a video called test2.6.mp4 or test2.6.avi, and this file is located in the project_folder/videos directory?

[carson2stoker]

No, sorry the sleap project is called test2.6.slp, the video is just called video.mp4.

[sronilsson]

Alright, is test2.6.slp a sleap project you are trying to import into SimBA? What you want to import, is the slp inference files for the videos, not the entire project. If you have used your test2.6.slp project to inferred the body-part predictions on video.mp4, then sleap should create a video.slp. Its the video.slp you want to import into simba.

[carson2stoker]

Yes that is what I am trying to do. So just to clarify, in the 'import videos into project folder' tab I should import the raw video and in the 'import tracking data' tab I should upload a video.slp file which is the sleap inference files? I have run inference and generated predictions on the whole video, but I don't have a file called video.slp. I do see one called test2.6.slp.mp4.avi though, would that be what I'm looking for?

[sronilsson]

Yes, what I am thinking, is that you use your test2.6.slp project, to select and annotate images etc, and ultimately build a pose-estimation model. Once that is done, you want to use it to the test2.6.slp project, to analyze new videos, such as video.mp4, so you run your inference on on video.mp4 using the model inside the test2.6.slp project. That should produce an output file, which holds all the body-part coordinates for video.mp4. I think they name that output file video.slp in sleap.

[carson2stoker]

Ah I figured it out. Sleap puts those .slp files generated during inference under a different directory called predictions and renames them. Once I found the right .slp file and renamed it to video.slp, it worked! It looks like the names have to match, which makes sense.

[sronilsson]

Great!