Slow network execute with bigger networks

[jessdtate]

It appears to be the scheduler. When I have ~ 30 modules, there is dramatic slowdown in network execution (maybe loading too), but it is in the prepping to execute stage that it spends a lot of time. I can wait over a minute waiting without actually executing code. Some networks that this is evident with are: IBBM2015/defib_FEM_eval.srn5 (35 modules), IBBM2015/data_visualization_NEEDS_MODULES.srn5 (40 modules), and Toolkits/FwdInvToolkit/Networks/potential-based-fem/forward_problem.srn5 (~29). The smallest of these takes ~30 sec on both my intel core 2 duo mac machines, and the biggest one can take ~2 min just to begin executing (1 min before it highlights all the modules and another minute to determine what needs to be execute)

These sizes of networks are when you can start doing interesting things in SCIRun. It is also worth noting that a few weeks ago it wasn't as slow to execute things (about the time of expanding the stop module execution was when it slowed down).

[dcwhite]

I believe it's the GUI thread and some changes I made there to get the coloring correct. The scheduling code won't ever be a bottleneck. I'd like you to test the bigger networks on a few other machines to get more benchmark data. In the meantime I will try to add some code to trace what is going on during these waiting periods.

[jessdtate]

I tried this with a few other machines. My Laptop is definitely the slowest. Most of the other machines, even with similar specs, are usually twice as fast at executing these networks. However, that still puts these networks ~20 sec to begin executing, plus some time to execute even when there is nothing to execute. I think there is a speedup with newer operating systems (10.9 and 10.10) but the hardware is usually better on those machines too. It may also be just more spread out, as it is shorter to begin executing, but take longer to execute nothing.

If the networks get just a little bigger (brain stimulator TDCS network, 48 modules), then there is addition slowdown that is non-linear (a lot slower than just 8 extra modules in smaller networks). gus is the fastest machine i tried and it takes about a minute execute no data on this network. I notice on gus that these steps were single threaded (only one processor was used and maxed out), as well as loading the network, which can also be very slow.

There also starts to be UI slowdowns at this level: delay in opening module ui, slow scrolling. gus doesn't really have this problem but every other machine does.

[dcwhite]

Thanks for the useful info. Work on this will fall under optimization which will have to wait until we're out of alpha, and in beta--probably a few months away at least.

[dcwhite]

@ajanson @SCIInstitute/butson-lab Here's the global network performance issue. Slotted for Q4, after Matlab and Python.

[dcwhite]

@jessdtate will add some more examples for me.

[dcwhite]

@ajanson showed me a ~100 module network, and the GUI re-execute performance was not too bad. Still waiting on @jessdtate's monster networks.

[github-actions[bot]]

Stale issue message

[dcwhite]

@jessdtate Still waiting on your monster networks.

[jessdtate]

This is hardward dependent, but I am surprised that I am not seeing this anymore in some of my old nets or in the ones I'm trying to run. I've attached a network with >200 modules, and it's executing fine on my my 10.12 and 10.15 machine. There may actually be a speed up from qt 5

sandbox_large.srn5.zip

[jessdtate]

Yes, running the example network in qt4 took ~ 30 sec to check the rexecute. This took about 2 seconds with qt5 on the same machine (10.12). Net loading and responsiveness was faster in qt5 (5.13) too.

[dcwhite]

Well, hurray! Maybe Qt5 will fix all our problems