TritonDataCenter / node-haproxy-log

very basic haproxy log parser
MIT License
0 stars 1 forks source link

reported malformed log line for entry from log file #1

Open davepacheco opened 6 years ago

davepacheco commented 6 years ago

I have this entry from an haproxy log file:

2018-03-13T19:05:29+00:00 710f85a9-6f13-4fcf-8f3f-a03c4a508714 haproxy[657137]: ::ffff:172.24.2.24:49297 [13/Mar/2018:19:05:29.503] https secure_api/be1 20/0/1/-1/233 -1 0 - - CH-- 49/32/48/15/0 0/0 "PUT /dap/stor/mlive/mlive_23/obj HTTP/1.1"

When I feed that to haplog, I get:

haplog: malformed log line: "2018-03-13T19:05:29+00:00 710f85a9-6f13-4fcf-8f3f-a03c4a508714 haproxy[657137]: ::ffff:172.24.2.24:49297 [13/Mar/2018:19:05:29.503] https secure_api/be1 20/0/1/-1/233 -1 0 - - CH-- 49/32/48/15/0 0/0 "PUT /dap/stor/mlive/mlive_23/obj HTTP/1.1""
davepacheco commented 5 years ago

Here's another case:

2019-02-06T22:15:12+00:00 3fce0562-b2b8-49fb-867a-966e20d45a9c haproxy[40068]: ::ffff:8.12.47.4:51346 [06/Feb/2019:22:15:04.184] https secure_api/be5 75/6005/-1/-1/8090 503 245 - - sC-- 142/103/125/11/+3 0/0 "GET /Joyent_Dev/public/builds/amon/master-20190206T201521Z-g08b2eb7/amon/amon-relay-master-20190206T201521Z-g08b2eb7.manifest HTTP/1.1"
dap@blinky ~ $ pbpaste | haplog
haplog: malformed log line: "2019-02-06T22:15:12+00:00 3fce0562-b2b8-49fb-867a-966e20d45a9c haproxy[40068]: ::ffff:8.12.47.4:51346 [06/Feb/2019:22:15:04.184] https secure_api/be5 75/6005/-1/-1/8090 503 245 - - sC-- 142/103/125/11/+3 0/0 "GET /Joyent_Dev/public/builds/amon/master-20190206T201521Z-g08b2eb7/amon/amon-relay-master-20190206T201521Z-g08b2eb7.manifest HTTP/1.1""
dap@blinky ~ $ 
trentm commented 5 years ago

I've a fix for the first one (it was the -1 for the status code):

$ echo '2018-03-13T19:05:29+00:00 710f85a9-6f13-4fcf-8f3f-a03c4a508714 haproxy[657137]: ::ffff:172.24.2.24:49297 [13/Mar/2018:19:05:29.503] https secure_api/be1 20/0/1/-1/233 -1 0 - - CH-- 49/32/48/15/0 0/0 "PUT /dap/stor/mlive/mlive_23/obj HTTP/1.1"' | ./bin/haplog | json
{
  "syslog_date": "2018-03-13T19:05:29.000Z",
  "syslog_hostname": "710f85a9-6f13-4fcf-8f3f-a03c4a508714",
  "pid": 657137,
  "client_ip": "172.24.2.24",
  "client_port": 49297,
  "accept_date": "13/Mar/2018:19:05:29.503",
  "frontend_name": "https",
  "backend_name": "secure_api",
  "server_name": "be1",
  "Tq": 20,
  "Tw": 0,
  "Tc": 1,
  "Tr": -1,
  "Tt_logasap": false,
  "Tt": 233,
  "status_code": -1,
  "bytes_read": 0,
  "termination_state": {
    "raw": "CH--",
    "termination_cause": "CLIENT_ABORT",
    "state_at_close": "WAITING_FOR_RESPONSE_HEADERS",
    "persistence_cookie_client": "N/A",
    "persistence_cookie_server": "N/A"
  },
  "actconn": 49,
  "feconn": 32,
  "beconn": 48,
  "src_conn": 15,
  "retries_redispatch": false,
  "retries": 0,
  "srv_queue": 0,
  "backend_queue": 0,
  "http_request": "PUT /dap/stor/mlive/mlive_23/obj HTTP/1.1"
}

and for the second one (it was the "+" on retries):

$ echo '2019-02-06T22:15:12+00:00 3fce0562-b2b8-49fb-867a-966e20d45a9c haproxy[40068]: ::ffff:8.12.47.4:51346 [06/Feb/2019:22:15:04.184] https secure_api/be5 75/6005/-1/-1/8090 503 245 - - sC-- 142/103/125/11/+3 0/0 "GET /Joyent_Dev/public/builds/amon/master-20190206T201521Z-g08b2eb7/amon/amon-relay-master-20190206T201521Z-g08b2eb7.manifest HTTP/1.1"' | ./bin/haplog | json
{
  "syslog_date": "2019-02-06T22:15:12.000Z",
  "syslog_hostname": "3fce0562-b2b8-49fb-867a-966e20d45a9c",
  "pid": 40068,
  "client_ip": "8.12.47.4",
  "client_port": 51346,
  "accept_date": "06/Feb/2019:22:15:04.184",
  "frontend_name": "https",
  "backend_name": "secure_api",
  "server_name": "be5",
  "Tq": 75,
  "Tw": 6005,
  "Tc": -1,
  "Tr": -1,
  "Tt_logasap": false,
  "Tt": 8090,
  "status_code": 503,
  "bytes_read": 245,
  "termination_state": {
    "raw": "sC--",
    "termination_cause": "BACKEND_DATA_TIMEOUT",
    "state_at_close": "PRE_BACKEND_CONNECTION",
    "persistence_cookie_client": "N/A",
    "persistence_cookie_server": "N/A"
  },
  "actconn": 142,
  "feconn": 103,
  "beconn": 125,
  "src_conn": 11,
  "retries_redispatch": true,
  "retries": 3,
  "srv_queue": 0,
  "backend_queue": 0,
  "http_request": "GET /Joyent_Dev/public/builds/amon/master-20190206T201521Z-g08b2eb7/amon/amon-relay-master-20190206T201521Z-g08b2eb7.manifest HTTP/1.1"
}

For which I've added "retries_redispatch": true,.

I've also added "Tt_logasap": false, for a possible '+' there per https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#8.2.3 docs for Tt.

trentm commented 5 years ago

CR: https://cr.joyent.us/#/c/5509/