Collapsible sidebar requires more clicks to use fields

[exabrial]

Long time graylog user here. I very much appreciate the work that goes into this project. Over the years, I've even contributed a few plugins, inputs, extractors, and dashboards.

With the 3.2 release, the streams UI took major changes. We now find it extraordinarily difficult to navigate and discover fields.

Is there a path to revert to 3.1 until the usability issues with 3.2 are fixed? We greatly need having the field list displayed and search dialogs.

Thank you for consideration.

[dennisoelkers]

Hey @exabrial, thanks for the feedback. Right now there is no downgrading path. However, you can try restoring a MongoDB backup from before the update and go back to 3.1.

Can you name exactly the usability issues that bother you? It will certainly help us to improve the product.

[exabrial]

I don't think we have any mongo backups unless it's automatic :( That's why we're hoping for a downgrade path. What changed in the mongodb config? Could we reverse it by hand?

Can you name exactly the usability issues that bother you? It will certainly help us to improve the product.

Yes, In general, everything that is now a popover instead of statically fixed to the left side of the screen has made the UI a bear to use.

The biggest issue is the lack of commonly used fields available in the stream on the left side. Those just be available all the time, without the need for a user to click over, cover up half the screen, select the field, then click over and close the popover. It was a piece of functionality that worked really well, and now what used to take 1 click takes at least 4. When you're hunting for bugs, you don't always know what you're looking for, so fast access to selecting/hiding fields is huge. That change along is burdensome beyond measure and has destroyed the usefulness of the stream view.

In addition, the search bar is now a "global search" when on the screen for a stream. The way the search bar used to work was intuitive. If I'm in a stream, I know my query will be limited to that stream. If I wanted to search multiple streams, I would use the search page, not a particular streams page. There's no reason to try and conflate this two features and it crowds the user interface.

We're kicking ourselves for upgrading, we just want the old view back. Apologies for the rough feedback, we're happy to use have you working on graylog. We're sad one of our favorite and most useful tools is now difficult to use. Unfortunately, my team also wants to look at different solutions because of this changes too.

[dennisoelkers]

Thanks for the extensive feedback, @exabrial!

The changes we did for 3.2 were based on the feature/improvement requests we heard the most from the community. Unfortunately what is better for one makes things more difficult for another. It is not in our interest to make life harder for long time users, to the contrary. I am pretty sure though that we can improve the product further and make it more useful for you and your team again. Therefore I would like to understand some of your points better:

• The collapsible sidebar is a controversial topic. Some prefer having more space on the screen and working with fields inline (fields and their values are supposed to be clickable everywhere and the actions you were triggering from the sidebar can happen even quicker than before when you are working with a single message), others want the sidebar to be visible all the time. We are currently discussing how to deal with this conflict and where the sweet spot is.

• I am not sure I understood your point about the stream search. When going to "Streams" and selecting a stream, the search which is started has the stream preselected. The search results and all further actions are scoped to that stream, unless you change the selection of streams. The reson to do it like this, is that pretty often a search is started on one stream, but at some the need arises to see data of a different stream side by side. In the past this was not possible, now you can just add a second stream, but you can still leave the selection confined to a single stream. Can you tell me a little bit more about why this is confusing/complicating things for you?

[exabrial]

Thanks for taking the time to listen, we appreciate it.

On the second point, when you're on a stream's page, the search box should limit the query to that stream by default; the search box shouldn't be a global search by default. The stream name is added there as a visual cue, but if the user just clicked on a stream name, why are they presented with a global search box? If they wanted to search all or multiple streams, they would go to the all messages streams and type a filter in to limit the stream scope. Essentially, when you're on a stream page, it should be dedicated to that stream, there shouldn't be a need to have the stream name in the search box, as that is implied.

[gpavinteractiv]

The old sidebar with checkboxes for each field was more convenient than the new one without the checkboxes. It's hard to know which fields have been selected now.

Also, it is not possible to uncheck the "message" field anymore (that saves a lot of space, as we are often only interested in a few fields and not the whole message). I tried to do message->"Remove from all tables" but that does not work.

[kmerz]

@gpavinteractiv before 3.2 it was not possible to remove the blue message line. This feature got introduced to 3.2. What do you mean with

Also, it is not possible to uncheck the "message" field anymore (that saves a lot of space, as we are often only interested in a few fields and not the whole message)

[gpavinteractiv]

@kmerz : On Graylog 3.1.3

With "message" checked (default) :

With "message" unchecked :

[dennisoelkers]

@gpavinteractiv: You can edit a message table widget and deselect the Show message in new row option. It looks like this:

Being able to select/deselect fields from the sidebar was very convenient. The reason we removed it is that you can now have an arbitrary amount of message tables which you can customize, so a central control for selecting/deselecting fields does not make sense anymore. For fields which occur in your messages, you can still add them to the current message table pretty quickly using the "Add to table" field action:

[exabrial]

I really think the old UI was far superior in this manner, it automatically showed what fields were present on the listed messages. The new UI makes you hunt for which fields contain data relavent. When problem solving, especially under time pressure as often graylog is used for, it was nice to have all the relevant fields displayed for you. If something was missing it was also made pretty obvious to the user.

For example, he's a stream that pulls haproxy logs from a syslog input, and the second pulls logback messages that have traveled over the ActiveMQ input plugin we wrote for Graylog. Only the important stuff is shown. That was an incredibly powerful tool and we're disappointed to see it gone.

[exabrial]

If anyone is looking for a downgrade path, here's how we're planning it. All of our inbound logs come over ActiveMQ, so we used a camel route to duplicate the incoming messages. We setup an older graylog 3.1 server and sent it a copy of the messages going to graylog 3.2. We really only look at last 14 days, so we'll tear down the graylog 3.2 server after we're confident we don't need any of the logs in that system.

I guess we'll stick with 3.1 until the usability issues are sorted out :( bummer. We're happy to help, test, and provide any feedback to the graylog developers. We love the product we're thankful for OSS like graylog.

[prcdpr]

I'm also not very happy with direction of UI changes in Graylog 3.2 :(

Two of the most frequently features I was using are now missing:

• always visible sidebar with discoverable fields
• quick values accessible with a couple of clicks

Can we have a downgrade path at least until next version rolls out that improves the UI?

[nosilver4u]

I'm returning to Graylog after having used it so long ago I don't even remember what the old interface looked like. But I'm finding myself having the same dilemma with the fields list. Currently we use Loggly, and fields can simply be collapsed on-demand. The sidebar doesn't collapse automatically, but you have the option to collapse it and get yourself that coveted screen space. I think this caters nicely to both crowds involved.

[linuspahl]

To provide an update on this topic, with 4.0 it will be possible to "pin" the sidebar:

This setting can be changed by clicking on the pin icon and will be saved as a user preference. The preference will be applied for all searches with the same type (the main search / saved searches or dashboards). Which means you can decide if you want to pin the sidebar for saved searches but collapse it for dashboards.

We hope this will improve the usability for the mentioned use cases. Thanks again for all your feedback!

[dennisoelkers]

Fixed in #8294.