Potential slower performance in 1d decode visualization

[jsoules]

Eric (@edeno) reported from Slack:

Hi all, We've noticed a slow down in the responsiveness in the 1D decoding visualizations lately. It is unclear to me whether this is related to the amount of data we are putting in or something else. I just wanted to flag it in case you have any thoughts. Here is one example: https://figurl.org/f?v=gs://figurl/spikesortingview-10&d=sha1://1452646ea5c6b863848a34dfdfa1d8996459ce1f&label=tonks20211028_.nwb

[jsoules]

Hi Eric, thanks for bringing this up. I'm seeing this is about a 50-minute recording with about 80 MB of data, is that right? Are you seeing the issue only on the new images, or are you experiencing slowness on older figures? (Can you give me an example of a figure that's performing well so I can get a sense of the comparison?) Are there particular scales or interactions where you're most noticing the issue?

[edeno]

Right. So I don't have any proof that it is slower than the older ones. Just that the interaction is much slower than say this example you posted: https://figurl.org/f?v=gs://figurl/spikesortingview-10&d=sha1://5ce6bd6dfce3932d2bde1a79b2a51ef39fab8e14&label=03f1-acausal-posterior&s={}

There might be many reasons for that. The posterior is twice as big. There are more plots.

As for scale, it does seem to be much worse on the more zoomed out view.

[jsoules]

It's possible the downsampling algorithm isn't aggressive enough for this particular scale, too--I don't know that the threshold I chose was all that principled; maybe need to revisit that...

If there are particular interactions (like "zoom to a 10 minute window from 20:00 to 30:00 and then pan a quarter of the figure width forward and backward") that we can focus on, that would be helpful too.

[edeno]

I think what you mentioned as an example interaction is perfect. The general scenario is probably people are starting at the zoomed out initial view and then trying to zoom in on a particular part of time to look at a behavior. So zoom and then pan around a bit to find the thing they care about. Then maybe zoom back out to check a different time period. The responsiveness when zoomed in is fine. It's just the initial lack of responsiveness that makes people think that there generally isn't any responsiveness. So, I agree that increasing the downsampling at that particular scale would help.

[jsoules]

So I've done some poking on this, and the issue doesn't seem to be with the 1D decode specifically. I tried isolating the 1D decode visualization and loading that--see https://figurl.org/f?v=gs://figurl/spikesortingview-10&d=sha1://957a15abe0fc7ead116a28cd2537c79ca4e9f6d7&label=tonks20211028_.nwb as an example, where I've just pulled out the 1d-decode part of the composite view (the sha1 in this Figurl is the same as the one referenced in the JSON from the link in the issue description) and that actually seems quite snappy.

For reference, the other attached views are:

• sha1://7ed756b3eaca4d54f75423b0a211d75e51d89fb3 ("Probability of State") --> https://figurl.org/f?v=gs://figurl/spikesortingview-10&d=sha1://7ed756b3eaca4d54f75423b0a211d75e51d89fb3&label=tonks20211028-PROB.nwb
• sha1://2e868423a9f3772ee043b59934499214b3a0d39d ("Speed") --> https://figurl.org/f?v=gs://figurl/spikesortingview-10&d=sha1://2e868423a9f3772ee043b59934499214b3a0d39d&label=tonks20211028-SPEED.nwb
• sha1://f8211fb096b1e75cada8ab6f2077a8f726b2840e ("Multiunit") --> https://figurl.org/f?v=gs://figurl/spikesortingview-10&d=sha1://f8211fb096b1e75cada8ab6f2077a8f726b2840e&label=tonks20211028-MULTIUNIT.nwb

Loading those individually, they actually all seem to be less performant for zooming and panning than the 1d decode is. I think we may need to revisit either the TimeseriesGraph component or the amount of data being graphed by these visualizations. As a first-order observation, while the performance improves upon zooming in, even at fairly low-visible-data levels it's still not super smooth (I've got the "Probability of State" chart down to showing about 15 seconds and there's noticeable stutter with panning) so it may just be the component itself, but there's probably also some downsampling or something that could be done. Not sure how to do it in a sufficiently generic way though.

[magland]

@jsoules that's great that you were able to narrow this down to an issue with TimeseriesGraph. The good news is that the TG widget should be a lot more straightforward to optimize/subsample compared with the more complex decode widget. I think I'll try my hand at addressing this using web workers.

[magland]

Here it is with web workers: https://figurl.org/f?v=gs://figurl/spikesortingview-10dev&d=sha1://1452646ea5c6b863848a34dfdfa1d8996459ce1f&label=tonks20211028_.nwb

Much more responsive, but there is a lag for the rendering. I tried various ways to do downsampling, but none of those was satisfying. My own opinion is to let there be a lag -- it's better to have the responsiveness.

You can compare with the non-web-worker version (which is still the production): https://figurl.org/f?v=gs://figurl/spikesortingview-10&d=sha1://1452646ea5c6b863848a34dfdfa1d8996459ce1f&label=tonks20211028_.nwb

@edeno Let me know what you think.

[jsoules]

Let's revisit this at our afternoon meeting and see if we can close out this issue.

(@magland The fix with the web workers is in production now, right?)

[magland]

Yes, it's in production. I'll just close the issue.

[magland]

Actually, I guess I'll reopen this, and keep it open until after our meeting.