Large Networks/Workspaces load very slowly.

[GoogleCodeExporter]

1. Create a large network, spec. 900 neurons & 81,000 synapses.
2. Save the workspace.
3. Clear & reload.

The loading process takes an unacceptably long time. Compared to the amount of 
time it takes to build a network like this from scratch (5 secs tops), loading 
takes 3 mins or more, during which time the GUI is completely unresponsive. 
Saving is also somewhat slow, but at ~15 secs, the save delay is not a 
concern... 

Stats derived from following the above steps with an i7-3770K x 8 @ 3.5GHz CPU, 
12 GB of RAM and a 128 GB SSD.

It is likely that the optimizations that allowed for 1000s of neurons and 
hundreds of thousands of synapses to run/build/respond in an acceptable amount 
of time so as to be easily manipulable by a user need to be extended to the 
loading process.

Original issue reported on code.google.com by ZachT...@gmail.com on 30 Jun 2014 at 6:30

Attachments:

• chaos.zip

[GoogleCodeExporter]

I agree this is a serious issue.    One stop-gap that can be implemented now is 
a progress bar so that the user has a sense of what they're dealing with in 
terms of save and load time, rather than the program appearing to be frozen.   
They can also be advised in the progress panel that scripting is a way of 
creating large networks without large persisted files and the associated wait 
times.   There are longer term solutions as well, which we've talked about 
(e.g. neuron groups and synapse groups with group level specifications), but I 
imagine that's for a future release.

Original comment by jeffyosh...@gmail.com on 30 Jun 2014 at 10:06

[GoogleCodeExporter]

I'd be happy to take a look into it myself. To be clear my posting something as 
an issue does not imply that I am unable or unwilling to fix the problem 
myself. It's just organizationally more sound than keeping TODO lists that I 
keep losing... 

Anyway, it could be an issue with XStream, it could also be an issue with the 
way the networks are being built... a bad data structure, redundant checks, 
etc. The other optimizations mostly amounted to fixing those two problems, so 
if we're lucky it's as simple as that. 

I like the idea of a progress bar in any case. However, if we're going to have 
a progress bar we need to run the de-serialization in a thread other than the 
EDT. As it stands it must be running in the EDT, because it freezes the UI, 
which I think is unacceptable... if we're going to make users wait, we should 
at least make the other parts of Simbrain accessible in the meantime. Of course 
the ideal solution is to not make them wait at all...

I'm attaching a performance report. Using visual VM I've confirmed that at 
least part of the issue is that XStream is running in the EDT. So the first 
part of fixing this is to make in run in another thread... this should be done 
regardless of the implementation/success of other optimizations. As you can see 
in the attachment xstream is where the bottleneck is happening... so perhaps my 
earlier theory about improper data structures or redundant checks within 
Simbrain code was incorrect.

Original comment by ZachT...@gmail.com on 30 Jun 2014 at 10:47

Attachments:

• PerformanceReport.png

[GoogleCodeExporter]

Original comment by ZachT...@gmail.com on 26 Jul 2014 at 9:48

• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Original comment by ZachT...@gmail.com on 12 Jan 2015 at 5:31

• Changed state: Fixed