tkuchiki / alp

Access Log Profiler
MIT License
671 stars 34 forks source link

Query string aggregation won't work. #4

Closed vzvu3k6k closed 8 years ago

vzvu3k6k commented 8 years ago

-q をつけると、QueryString の key? も含めて集計、 つけなければ QueryString 抜きで集計します。

http://tkuchiki.hatenablog.com/entry/2015/09/25/130900

According to this explanation, alp should ignore QueryString in aggregation when -q (or --query-string) is not given, but it won't.

Example

Using alp/access.log as an input.

$ alp --version
0.3.0
$ alp -f access.log
+-------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+--------+------------------------------+
| COUNT |  MIN  |  MAX  |  SUM  |  AVG  |  P1   |  P50  |  P99  | STDDEV | MIN(BODY) | MAX(BODY) | SUM(BODY) | AVG(BODY) | METHOD |             URI              |
+-------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+--------+------------------------------+
|     2 | 0.057 | 0.057 | 0.114 | 0.057 | 0.057 | 0.057 | 0.057 |  0.000 |    12.000 |    12.000 |    24.000 |    12.000 | POST   | /foo/bar?token=xxx&uuid=1234 |
|     2 | 0.123 | 0.123 | 0.246 | 0.123 | 0.123 | 0.123 | 0.123 |  0.000 |    56.000 |    56.000 |   112.000 |    56.000 | GET    | /foo/bar?token=zzz           |
|     1 | 0.135 | 0.135 | 0.135 | 0.135 | 0.135 | 0.135 | 0.135 |  0.000 |    15.000 |    15.000 |    15.000 |    15.000 | GET    | /diary/entry/1234            |
|     3 | 0.100 | 0.234 | 0.434 | 0.145 | 0.100 | 0.100 | 0.100 |  0.063 |    34.000 |    34.000 |   102.000 |    34.000 | POST   | /foo/bar?token=yyy           |
|     1 | 0.234 | 0.234 | 0.234 | 0.234 | 0.234 | 0.234 | 0.234 |  0.000 |    34.000 |    34.000 |    34.000 |    34.000 | POST   | /hoge/piyo?id=yyy            |
|     1 | 0.432 | 0.432 | 0.432 | 0.432 | 0.432 | 0.432 | 0.432 |  0.000 |    30.000 |    30.000 |    30.000 |    30.000 | GET    | /diary/entry/5678            |
+-------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+--------+------------------------------+

The expected output is:

+-------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+--------+-------------------+
| COUNT |  MIN  |  MAX  |  SUM  |  AVG  |  P1   |  P50  |  P99  | STDDEV | MIN(BODY) | MAX(BODY) | SUM(BODY) | AVG(BODY) | METHOD |        URI        |
+-------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+--------+-------------------+
|     2 | 0.123 | 0.123 | 0.246 | 0.123 | 0.123 | 0.123 | 0.123 |  0.000 |    56.000 |    56.000 |   112.000 |    56.000 | GET    | /foo/bar          |
|     1 | 0.135 | 0.135 | 0.135 | 0.135 | 0.135 | 0.135 | 0.135 |  0.000 |    15.000 |    15.000 |    15.000 |    15.000 | GET    | /diary/entry/1234 |
|     5 | 0.057 | 0.234 | 0.548 | 0.110 | 0.057 | 0.057 | 0.100 |  0.065 |    12.000 |    34.000 |   126.000 |    25.200 | POST   | /foo/bar          |
|     1 | 0.234 | 0.234 | 0.234 | 0.234 | 0.234 | 0.234 | 0.234 |  0.000 |    34.000 |    34.000 |    34.000 |    34.000 | POST   | /hoge/piyo        |
|     1 | 0.432 | 0.432 | 0.432 | 0.432 | 0.432 | 0.432 | 0.432 |  0.000 |    30.000 |    30.000 |    30.000 |    30.000 | GET    | /diary/entry/5678 |
+-------+-------+-------+-------+-------+-------+-------+-------+--------+-----------+-----------+-----------+-----------+--------+-------------------+

Cause

Since 60d9f56, uri is escaped with url.QueryEscape to handle an invalid one, but this breaks a uri because what QueryEscape does is to percent-encode the given string.

fmt.Println(url.QueryEscape("/foo/bar?token=xxx&uuid=1234"))
// => %2Ffoo%2Fbar%3Ftoken%3Dxxx%26uuid%3D1234

Solution

This pull request makes alp just try parsing a uri and skip the record if its uri can't be parsed.

tkuchiki commented 8 years ago

@vzvu3k6k Thank you for your Pull Request. Great! 👍

tkuchiki commented 8 years ago

@vzvu3k6k v0.3.1 released!