Plot threshold is taking too long

[Urimons]

Describe the bug Hello, I have finished most steps with SimBA (after importing CSVs from DLC) and now I got to the "plot threshold" stage. It seems to work fine, although it takes forever (more accurately approximately a day for a video of 30min and 30fps = 54,000 f). Other stages like "Gantt plot" or "Data plot" took much less (a day or two for all the videos). What do you think can be the problem and how can I fix it?
Thanks in advance :)

To Reproduce Steps to reproduce the behavior:

1. Go to "Visualizations"
2. Click on 'Plot threshold'...

Screenshots

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Python Version: 3.7.6
• Are you using anaconda? yes

Additional context Add any other context about the problem here.

[sronilsson]

Hi @Urimons! Thanks for reporting.

If you update simba with pip install simba-uw-tf-dev --upgrade, do you still see the same runtime issue?

A bit of background: most plotting code in SimBA has had speed issues, it's creating 54k graphs and can be expected to be slow as it runs on single core. However, it shouldn't be noticable slower than other graph functions... I'm working on getting in multiprocessing, but I have not got very far. In fact I have only really completed multiprocessing of Gantt plot.

If you update Simba and still see speed issue, please let me know and I will do the probability plots next.

[Urimons]

Thanks! I upgraded and it seems to run a bit faster, it runs 216K frames (4 videos) per day intead of 54K frames (1 video) per day that was before, still almost 6-7 times slower than other plots...

[sronilsson]

Thanks @Urimons - I've inserted multiprocessing for probability visualizations: does it go any quicker if you do pip install simba-uw-tf-dev --upgrade to version 0.99.1?

The tricky bit I find for creating videos/frames on many cores is dynamically avoiding MemoryErrors and overloading CPUs when I can't anticipate what machine the users have, so now I am conservative in the multiprocessing code. Should nevertheless be noticable faster. Please let me know how it goes.

[Urimons]

Now it runs untill it actually surpass the amount of existing frames in a single video and then stuck...

[sronilsson]

Thanks @Urimons, the count shouldn't matter, but will fix.

The error could be that conda ffmpeg isn't installed. If you type conda install -c conda-forge ffmpeg and try, does that fix it?