Fix router log parsing when `disable_log_source_ip = true` and fix routing API log parsing

[peterellisjones]

Hi Logsearch-for-cloudfoundry team,

This PR fixes two issues:

Router log parse failure when disable_log_source_ip = true

When the gorouter is configured to not log source IPs, the source IP field in the router logs is replaced with -, rather than the usual host-port pair. Logsearch-for-cloudfoundry is currently unable to parse these logs, which result in them being tagged with fail/cloudfoundry/app-rtr/grok and the rest of the fields not being extracted properly.

This PR replaces the source IP log parsing code for cf 252+ from:

%{IPORHOST:[rtr][src][host]}:%{POSINT:[rtr][src][port]:int}

to:

(%{IPORHOST:[rtr][src][host]}:%{POSINT:[rtr][src][port]:int}|-)

Supporting logs where source IP logging is disabled.

Router log parse failure for routing API logs

Routing API logs have the following format:

redacted.hostname.com - [2019-04-02T11:55:36.305+0000] "GET /routing/v1/router_groups HTTP/1.1" 200 0 114 "-" "HTTPClient/1.0 (2.8.3, ruby 2.4.5 (2018-10-18))" "-" "192.168.2.60:3000" x_forwarded_for:"192.168.0.8" x_forwarded_proto:"https" vcap_request_id:"6f68924a-b04a-4796-4375-b9c904e7a6c4" response_time:0.003233121 app_id:"routing_api" app_index:"-" x_b3_traceid:"e1145742e1755f33" x_b3_spanid:"e1145742e1755f33" x_b3_parentspanid:"-"

For the routing API logs, app_index is -, not an integer so the parsing rule fails.

This PR replaces the app_index log parsing code for cf 252+ from:

%{BASE10NUM:[rtr][app][index]:int}

to:

(%{BASE10NUM:[rtr][app][index]:int}|-)

Supporting logs where app index is -.

---

We have tested this manually by editing the log parsing code on an ingestor, restarting and verifying that the router logs are parsed correctly.

kind regards,

Pete

[CLAassistant]

All committers have signed the CLA.