nginx+modsecurityv3 config mlogc by SecAuditLog failed

[mxmkeep]

Hi~ I install nginx+modsecurityv3 and running well. and now i want to send modsec log by mlogc, but nginx testing failed. Where do I configure wrong? thanks for you help。

nginx error.log

2018/05/14 16:45:33 [notice] 17847#0: ModSecurity-nginx v1.0.0
2018/05/14 16:45:33 [emerg] 17847#0: "modsecurity_rules_file" directive Rules error. File: /home/mxm/code/trunk/nginx/conf/modsecurity/modsecurity.conf. Line: 176. Column: 103. Invalid input:  SecAuditLog "|/home/mxm/code/trunk/nginx/mlogc /home/mxm/code/trunk/nginx/conf/modsecurity/mlogc.conf" in /home/mxm/code/trunk/nginx/conf/nginx_prod.conf:137

modsecurity.conf

# Enable ModSecurity, attaching it to every transaction. Use detection
# only to start with, because that minimises the chances of post-installation
# disruption.
#
SecRuleEngine DetectionOnly
#SecRuleEngine On

# -- Request body handling ---------------------------------------------------

# Allow ModSecurity to access request bodies. If you don't, ModSecurity
# won't be able to see any POST parameters, which opens a large security
# hole for attackers to exploit.
#
SecRequestBodyAccess On

# Enable XML request body parser.
# Initiate XML Processor in case of xml content-type
#
SecRule REQUEST_HEADERS:Content-Type "(?:application(?:/soap\+|/)|text/)xml" \
     "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"

# Enable JSON request body parser.
# Initiate JSON Processor in case of JSON content-type; change accordingly
# if your application does not use 'application/json'
#
SecRule REQUEST_HEADERS:Content-Type "application/json" \
     "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"

# Maximum request body size we will accept for buffering. If you support
# file uploads then the value given on the first line has to be as large
# as the largest file you are willing to accept. The second value refers
# to the size of data, with files excluded. You want to keep that value as
# low as practical.
#
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072

# Store up to 128 KB of request body data in memory. When the multipart
# parser reachers this limit, it will start using your hard disk for
# storage. That is slow, but unavoidable.
#
#SecRequestBodyInMemoryLimit 131072

# What do do if the request body size is above our configured limit.
# Keep in mind that this setting will automatically be set to ProcessPartial
# when SecRuleEngine is set to DetectionOnly mode in order to minimize
# disruptions when initially deploying ModSecurity.
#
SecRequestBodyLimitAction Reject

# Verify that we've correctly processed the request body.
# As a rule of thumb, when failing to process a request body
# you should reject the request (when deployed in blocking mode)
# or log a high-severity alert (when deployed in detection-only mode).
#
SecRule REQBODY_ERROR "!@eq 0" \
"id:'200002', phase:2,t:none,log,deny,status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"

# By default be strict with what we accept in the multipart/form-data
# request body. If the rule below proves to be too strict for your
# environment consider changing it to detection-only. You are encouraged
# _not_ to remove it altogether.
#
SecRule MULTIPART_STRICT_ERROR "!@eq 0" \
"id:'200003',phase:2,t:none,log,deny,status:400, \
msg:'Multipart request body failed strict validation: \
PE %{REQBODY_PROCESSOR_ERROR}, \
BQ %{MULTIPART_BOUNDARY_QUOTED}, \
BW %{MULTIPART_BOUNDARY_WHITESPACE}, \
DB %{MULTIPART_DATA_BEFORE}, \
DA %{MULTIPART_DATA_AFTER}, \
HF %{MULTIPART_HEADER_FOLDING}, \
LF %{MULTIPART_LF_LINE}, \
SM %{MULTIPART_MISSING_SEMICOLON}, \
IQ %{MULTIPART_INVALID_QUOTING}, \
IP %{MULTIPART_INVALID_PART}, \
IH %{MULTIPART_INVALID_HEADER_FOLDING}, \
FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"

# Did we see anything that might be a boundary?
#
SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \
"id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"

# PCRE Tuning
# We want to avoid a potential RegEx DoS condition
#
SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000

# Some internal errors will set flags in TX and we will need to look for these.
# All of these are prefixed with "MSC_".  The following flags currently exist:
#
# MSC_PCRE_LIMITS_EXCEEDED: PCRE match limits were exceeded.
#
SecRule TX:/^MSC_/ "!@streq 0" \
        "id:'200005',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'"

# -- Response body handling --------------------------------------------------

# Allow ModSecurity to access response bodies. 
# You should have this directive enabled in order to identify errors
# and data leakage issues.
# 
# Do keep in mind that enabling this directive does increases both
# memory consumption and response latency.
#
SecResponseBodyAccess On

# Which response MIME types do you want to inspect? You should adjust the
# configuration below to catch documents but avoid static files
# (e.g., images and archives).
#
SecResponseBodyMimeType text/plain text/html text/xml

# Buffer response bodies of up to 512 KB in length.
SecResponseBodyLimit 524288

# What happens when we encounter a response body larger than the configured
# limit? By default, we process what we have and let the rest through.
# That's somewhat less secure, but does not break any legitimate pages.
#
SecResponseBodyLimitAction ProcessPartial

# -- Filesystem configuration ------------------------------------------------

# The location where ModSecurity stores temporary files (for example, when
# it needs to handle a file upload that is larger than the configured limit).
# 
# This default setting is chosen due to all systems have /tmp available however, 
# this is less than ideal. It is recommended that you specify a location that's private.
#
SecTmpDir /tmp/

# The location where ModSecurity will keep its persistent data.  This default setting 
# is chosen due to all systems have /tmp available however, it
# too should be updated to a place that other users can't access.
#
SecDataDir /tmp/

# -- File uploads handling configuration -------------------------------------

# The location where ModSecurity stores intercepted uploaded files. This
# location must be private to ModSecurity. You don't want other users on
# the server to access the files, do you?
#
#SecUploadDir /opt/modsecurity/var/upload/

# By default, only keep the files that were determined to be unusual
# in some way (by an external inspection script). For this to work you
# will also need at least one file inspection rule.
#
#SecUploadKeepFiles RelevantOnly

# Uploaded files are by default created with permissions that do not allow
# any other user to access them. You may need to relax that if you want to
# interface ModSecurity to an external program (e.g., an anti-virus).
#
#SecUploadFileMode 0600

# -- Debug log configuration -------------------------------------------------

# The default debug log configuration is to duplicate the error, warning
# and notice messages from the error log.
#
SecDebugLog /var/log/mod_debug.log
SecDebugLogLevel 9

# -- Audit log configuration -------------------------------------------------

# Log the transactions that are marked by a rule, as well as those that
# trigger a server error (determined by a 5xx or 4xx, excluding 404,  
# level response status codes).
#
SecAuditEngine RelevantOnly
#SecAuditLogRelevantStatus "^(?:5|4(?!04))"

# Log everything we know about a transaction.
SecAuditLogParts ABIJDEFHZ

# Use a single file for logging. This is much easier to look at, but
# assumes that you will use the audit log only ocassionally.
#
SecAuditLogType Serial
#SecAuditLog /var/log/modsec_audit.log
SecAuditLog "|/home/mxm/code/trunk/nginx/mlogc /home/mxm/code/trunk/nginx/conf/modsecurity/mlogc.conf"
#SecAuditLog "|/usr/bin/mlogc /etc/mlogc.conf"

# Specify the path for concurrent audit logging.
SecAuditLogStorageDir /var/log/mlogc/data

# -- Miscellaneous -----------------------------------------------------------

# Use the most commonly used application/x-www-form-urlencoded parameter
# separator. There's probably only one application somewhere that uses
# something else so don't expect to change this value.
#
SecArgumentSeparator &

# Settle on version 0 (zero) cookies, as that is what most applications
# use. Using an incorrect cookie version may open your installation to
# evasion attacks (against the rules that examine named cookies).
#
SecCookieFormat 0

# Specify your Unicode Code Point.
# This mapping is used by the t:urlDecodeUni transformation function
# to properly map encoded data to your language. Properly setting
# these directives helps to reduce false positives and negatives.
#
SecUnicodeMapFile unicode.mapping 20127

# Improve the quality of ModSecurity by sharing information about your
# current ModSecurity version and dependencies versions.
# The following information will be shared: ModSecurity version,
# Web Server version, APR version, PCRE version, Lua version, Libxml2
# version, Anonymous unique id for host.
SecStatusEngine On

mlogc.conf

# Required configuration
#   At a minimum, the items in this section will need to be adjusted to
#   fit your environment.  The remaining options are optional.
##########################################################################

# Points to the root of the installation. All relative
# paths will be resolved with the help of this path.
CollectorRoot       "/var/log/mlogc"

# ModSecurity Console receiving URI. You can change the host
# and the port parts but leave everything else as is.
#ConsoleURI          "https://CONSOLE_IP_ADDRESS:8888/rpc/auditLogReceiver"
ConsoleURI          "http://192.168.47.103:4000/MSALParser/web/rpc.php"
# Sensor credentials
#SensorUsername      "SENSOR_USERNAME"
#SensorPassword      "SENSOR_PASSWORD"

# Base directory where the audit logs are stored.  This can be specified
# as a path relative to the CollectorRoot, or a full path.
LogStorageDir       "data"

# Transaction log will contain the information on all log collector
# activities that happen between checkpoints. The transaction log
# is used to recover data in case of a crash (or if Apache kills
# the process).
TransactionLog      "mlogc-transaction.log"

# The file where the pending audit log entry data is kept. This file
# is updated on every checkpoint.
QueuePath           "mlogc-queue.log"

# The location of the error log.
ErrorLog            "mlogc-error.log"

# The location of the lock file.
LockFile            "mlogc.lck"

# Keep audit log entries after sending? (0=false 1=true)
# NOTE: This is required to be set in SecAuditLog mlogc config if you
# are going to use a secondary console via SecAuditLog2.
KeepEntries         0

##########################################################################
# Optional configuration
##########################################################################

# The error log level controls how much detail there
# will be in the error log. The levels are as follows:
#   0 - NONE
#   1 - ERROR
#   2 - WARNING
#   3 - NOTICE
#   4 - DEBUG
#   5 - DEBUG2
#
ErrorLogLevel       3

# How many concurrent connections to the server
# are we allowed to open at the same time? Log collector uses
# multiple connections in order to speed up audit log transfer.
# This is especially needed when the communication takes place
# over a slow link (e.g. not over a LAN).
MaxConnections      10

# How many requests a worker will process before recycling itself.
# This is to help prevent problems due to any memory leaks that may
# exists.  If this is set to 0, then no maximum is imposed. The default
# is 1000 requests per worker (the number of workers is controlled by the
# MaxConnections limit).
MaxWorkerRequests   1000

# The time each connection will sit idle before being reused,
# in milliseconds. Increase if you don't want ModSecurity Console
# to be hit with too many log collector requests.
TransactionDelay    50

# The time to wait before initialization on startup in milliseconds.
# Increase if mlogc is starting faster then termination when the
# sensor is reloaded.
StartupDelay        5000

# How often is the pending audit log entry data going to be written
# to a file. The default is 15 seconds.
CheckpointInterval  15

# If the server fails all threads will back down until the
# problem is sorted. The management thread will periodically
# launch a thread to test the server. The default is to test
# once in 60 seconds.
ServerErrorTimeout  60

# The following two parameters are not used yet, but
# reserved for future expansion.
# KeepAlive         150
# KeepAliveTimeout  300

# When set to '0', mlogc will validate the certificate and the whole
# chain, the root certificate most be trusted. If this check fail the
# connection will be dropped. To ignore the SSL checks, set InsecureNoCheckCert
# to '1'
InsecureNoCheckCert 1

[mxmkeep]

i can't find mlogc in ModSecurity-3.0.2, so i make it by modsecurity-2.9.2

[victorhora]

@mxmkeep using the SecAuditLog "|/path/to/mlogc /path/to/mlogc.conf" syntax is currently not supported on libModSecurity (v3).

If you need to use mlogC you have to open it using an external process and not relying on ModSecurity. You may also want to check #1589.

Alternatively you may also try with the SecAuditLogType HTTPS directive which is available on v3.

[mxmkeep]

@victorhora thanks for you help, and sending log successful. by the way, i was confuse of TBI and TBD, full word are? Supported on libModSecurity: TBI Supported on libModSecurity: TBD

[victorhora]

@mxmkeep

TBI = To be implemented TBD = To be discussed (in the sense that we are not sure if this will be implemented or deprecated on the new version)

:)

[kwaazaar]

I tried the SecAuditLogType HTTPS, but with no results. No HTTP call is being performed at all. No entries in the modsecurity debug log (I only see rule processing output) and nothing in the nginx logfiles.