Javascript? Blocking Results Pages for 6+ seconds and freezing the Browser

[NickMeves]

Web-Interface Version: 1.0.1

On pages with a total results historgram (Streams and Search) there is a 1 second delay for the browser to load the Histogram. Then the browser freezes for 7-8 seconds before the Saved Searches load (saved searches rest API response had already come well before this 7-8 seconds). Saved searches has no delay loading on pages without a Total Results histogram.

All Rest API calls are returning within 200ms. During the browser freeze no other background API actions occur (/a/system/throughput won't start its calls until after the block is done).

This seems to occur no matter what the data size (5 minute search, 30 day search, same block and browser freeze).

Oddly, this only appears to happen for Admin users. Reader users don't experience this on Stream pages, while Admin users do looking at the same stream have the blocked page for 7-8 seconds.

[edmundoa]

Hi,

Thank you for reporting this issue.

I have seen some delay loading the histogram in certain situations using AWS, but it may also happen on a slow machine or when the data set is really big. What I have never seen is that 7-8 seconds delay loading saved searches when a histogram is present.

Which browser and operating system are you using? It would be also helpful to know if you have any other charts in the search result page, how many messages you receive per second, and where your Graylog web interface is located (just knowing it it's in a local network or somewhere on the Internet is enough).

[NickMeves]

Sorry for the delay, I wanted to dig into this sooner but other issues came up.

This is on Firefox on Windows. 5 second freeze happens whether 1 event in last 5 minutes are graphed or 100,000,000 events in the last 30 days are graphed. We get about 1000-2000 events per second. This system is on our internal network.

[edmundoa]

Don't worry, thank you for giving more information about it!

I was looking into it and I could reproduce the problem on IE, the page load on Firefox was much better for me. We are in the middle of some deep changes in the web interface and I will take care of reviewing this once we are almost done, so we fix this issue on 1.1.

[NickMeves]

I tried to hop through the javascript with the debugger to find the culprit yesterday, but I didn't have luck yet. It didn't seem to be graph related, that aspect of document.ready was quite snappy.

Some more data points for us: Our Firefox on Windows is old - we are getting outdated browser notifications throughout the web (Thanks corporate!) This works perfectly as an Admin on Safari on a Mac. Firefox on a Mac (newer version) has a slight delay (2-3 seconds before saved search appears).

I'm still baffled by the Admin vs Reader differences. I will try to make a test reader that is more beefy (all streams and dashboards assigned) to see if it starts getting the delay to isolate what is causing this for us.

[NickMeves]

Found it!

The freeze is caused by having too many Dashboard edit permissions (merely read doesn't cause the delay). So our admins have access to all of them (~20 for us).

The freeze time is proportional to how many dashboards the user has edit permissions for.

I guess this is loaded to give a list of available dashboards when the user click on the dashboard button?

[NickMeves]

Yep, I think the freeze is in dashboards.js.

This function and first jQuery call:

function applyDashboardsToAllSelectors() { var dashboardSelectors = $(".dashboard-selector[data-widget-type]");

loads up 1300 objects into dashboardSelectors for us. x20 dashboards, that's a lot of calls in the .each() callback a little later in the call.

Maybe if each dashboard icon loads the global dashboards list onclick this can get better (loading 1 as needed rather than 1000+ on page load slowing things down)?

Semi-related: What's the reason for loading all the unneeded fields for the current search with display:none? This is adding 2-10x the fields needed to any given query - which compounds the issue of the dashboard bug since they all have the class dashboard-selector.

[edmundoa]

Hey,

Thank you for looking into it and providing so much information! :+1:

My impression was also that applyDashboardsToAllSelectors() was the cause of most problems there. Will take a look into it, but most likely the fix will have to wait for 1.1. In the meantime, could you please try to reduce the number of fields loaded by the web interface and see if helps? You can do that by lowering the value of field_list_limit in the Graylog web interface configuration (the default is 100).

The main reason for loading all fields in the search result page is that sometimes you want to work with fields that are not in the current search. The truth is that we don't load all fields, only the number set in the field_list_limit configuration option.

[NickMeves]

Thanks for the field_list_limit tip! I'll be sure to tune that.

Deleting some old dashboards and getting the count down from 20 helped alot as well.

[edmundoa]

We rewrote most of the search result page and this issue should be gone. We will release a beta version soon, it would be great if you could take a look and see if it helped :)