search

catmaid / CATMAID

Collaborative Annotation Toolkit for Massive Amounts of Image Data
https://catmaid.org
GNU General Public License v3.0
182 stars 56 forks source link

Overhead in node creation #1628

Open schlegelp opened 6 years ago

schlegelp commented 6 years ago

Not sure whether to open a new issue for this but I've noticed similar issues to #1594

When creating a new node (as child of existing node) /node/list is called twice again. Looking at my original post in this issue, this used to not be the case.

I've run a quick test to see which actions appear to cause overhead:

  1. New node (as child of existing node): 2x /node/list
  2. Selecting a single-node neuron: 1x skeleton/annotationlist (seems unnecessary?)
  3. Deleting connector: 2x /node/list (which I think is once for the connector and once for the presynaptic link
  4. Selecting a connector: 2x connector/skeletons and 2x skeleton/annotationlist

If updating the tracing overlay after user action is essential, would it be possible to make that update part of the response to .../create, .../delete, respectively?

I'm guessing these things require major changes but there are more cases where combining the queries would make a huge difference: e.g. joining an orphan node onto a neuron involves 6 queries (info -> permissions -> permissions -> node_count -> join -> list) which totals to about 1.2-1.8 seconds from the UK.

tomka commented 6 years ago

I completely agree, thee is room for optimizations! Thanks for listing potentially problematic requests! Most of them are indeed not needed, but happen as part of event handlers in the front-end (for events like "node created", etc.), it should therefore be possible to prevent most of those requests.

Unfortunately, the fix for #1619 caused the overlay updates during node creation (1) to reappear. I am in the middle of refactoring how individual user actions and their sub-actions in the tracing overlay are queued so that I can prevent those unnecessary queries more robustly.

schlegelp commented 6 years ago

Thanks for looking into this, @tomka. We're really appreciating every millisecond you can shave off.

tomka commented 6 years ago

The main bottlenecks 1. and 3. are fixed including the prevention of a single /node/list call after treenode deletion.

tomka commented 6 years ago

From 4. I just removed one request to each connector/skeletons and skeleton/annotationlist. It turned out nodes were activated twice when clicked.

The remaining requests in 2. (skeleton/annotationlist) and 4. (one of both connector/skeletons and skeleton/annotationlist don't necessarily have to be done, but they are currently used to update the neuron/connector name display in the upper right corner of a stack viewer. Given that that typical neuron name components don't change very often, we could probably could use a cached version of this information (after an initial request), but then again, those queries shouldn't take long. What timings do you see for them?

schlegelp commented 6 years ago

Each query is typically 200-300ms regardless of its type just because of latency between here and Janelia.