Add a panel to display query strings

[tumluliu]

I would like to have a stats graph for the querystring parameters in each requests. Such kind of fine-grained metrics are very important for us. Is it possible to implement a filter/plugin/extension for this in goaccess, as well as a panel on the report page? Thanks!

[allinurl]

515 will give users the ability to add custom panels and #117 will give the ability to filter the dataset. However, since this request appears to be fairly straightforward, I can add this panel to the report.

[allinurl]

BTW, should the query string be stripped from the request and add it to a different panel, or keep it in the request and add it to the other panel? Thoughts?

[tumluliu]

it would be great if you can add that panel. thanks so much, @allinurl ! For my use case, it would be better to show the stats of query strings in a separate panel. Besides, I hope I can customize which query string parameters should be included in the stats. Is that possible?

[allinurl]

Until #117 is implemented, you will need to do some preprocessing with tools such as sed, awk, grep, etc to filter out the query string. For instance,

tail -f -n +0 access.log | grep --line-buffered '?myqstr=value' | goaccess -o report.html --real-time-html -

[tumluliu]

with the grep --line-buffered '?myqstr=value' filter, I can get all the requests with myqstr=value. That's true, but a little bit different from what I really want which is more like the GROUP BY in sql.

Let's say my query string's pattern is like this:

?key=somevalue

where somevalue is an enumerator whose values can be [val_1, val_2, ... val_n].

What I want to see is a histogram panel on the number of requests for each val_i. Is it possible to do that with grep/sed/awk?

[tumluliu]

And furthermore, I would like to have multi-level filtering for different endpoints. E.g. if I have such requests:

/service1?key=somevalue
/service2?key=somevalue
/service3?key=somevalue
/service2?key=somevalue

I want to have the histogram stats of query string parameter key, but categorized by different service endpoints. I guess I have to write an awk script...

[allinurl]

Sounds like some pre-processing will do it, but let me add the query string panel first and push upstream so you can play with the preprocessing.

[tumluliu]

looking forward to that panel :)

I have already written some awk scripts

[t0mtaylor]

@allinurl where is the query string panel? I cannot see it in the list of panels on the man page? https://goaccess.io/man

[allinurl]

@t0mtaylor There is no dedicated query string panel. However, you can use the %q placeholder to append the query string to the request. If you would like to display the query string on its own panel, you can use the %e placeholder instead of %q. The query string will appear under the users panel (assuming it is not in use).