search

mtrimarchi / StunnelGrok4Datadog

stunnel service Datadog Grok implementation
2 stars 0 forks source link

Rules are different from ones created by the Engineering Team at Datadog #1

Open mtrimarchi opened 5 years ago

mtrimarchi commented 5 years ago

Datadog has presented his full set of rules starting from my examples reported to them by ticket/feedback. I'll try to make some tests in order to change my set and optimize my solution.

These are their actual rules on the Grok Parser "Parsing Stunnel Proxy logs":

Parsing rules

stunnel.service.connected_remote_server_from %{_date_stunnel} LOG%{_log_status}\[%{_session_id}\]\: Service \[%{_service_name}\] connected remote server from %{_local_ip}\:%{_local_port}

stunnel.service.accepted_connection_from %{_date_stunnel} LOG%{_log_status}\[%{_session_id}\]\: Service \[%{_service_name}\] accepted connection from %{_client_ip}\:%{_client_port}

stunnel.certificate.accepted %{_date_stunnel} LOG%{_log_status}\[%{_session_id}\]\: Certificate accepted at depth\=%{_cert_depth}\: %{_cert_info}

stunnel.connection.closed_reset %{_date_stunnel} LOG%{_log_status}\[%{_session_id}\]\: Connection (closed|reset)\: %{_byte_sent_to_ssl} byte\(s\) sent to SSL\, %{_byte_sent_to_socket} byte\(s\) sent to socket

stunnel.s_connect %{_date_stunnel} LOG%{_log_status}\[%{_session_id}\]\: (s_connect|transfer)\: (connect|connected|connecting|s_poll_wait) %{_backend_ip}\:%{_backend_port}(\: %{_error_message})?

stunnel.fallback %{_date_stunnel} LOG%{_log_status}\[%{_session_id}\]\: %{_error_message}

Helper rules

_backend_ip  %{ipOrHost:network.backend.ip}
_backend_port %{number:network.backend.port}
_byte_sent_to_socket %{integer:network.bytes_socket}
_byte_sent_to_ssl %{integer:network.bytes_ssl}
_cert_depth %{integer:proxy.certificate.depth}
_cert_info %{data:proxy.certificate.info:keyvalue}
_client_ip %{ipOrHost:network.client.ip}
_client_port %{number:network.client.port}$
_date_stunnel %{date("yyyy.MM.dd HH:mm:ss"):date}
_local_ip %{ipOrHost:network.local.ip}
_local_port %{number:network.local.port}
_log_status %{integer:level}
_session_id %{data:session_id}
_service_name (%{notSpace:proxy.service_name}-%{number:proxy.service_port}|%{notSpace:proxy.service_name})
_error_message %{data:error.message}

Final considerations

As we can see stunnel.log_status is now replaced with level, so the Status Remapper has to be fixed up with the new variable name.

Date Remapper has no differences because the variable date is untouched except for the TZ's information.

mtrimarchi commented 5 years ago

Task List