When deploying an ELB Service Log Pipeline, a Lambda Function named xxxxxx-LogProcessorFn is deployed with the description: "Function to process and load ELB logs into OpenSearch"
Inside this Lambda, there's the file util/log_parser.py
In the ELB Python class, the method parse (line 88) checks if a line from the log file matches a specific RegEx pattern.
If it doesn't, the line is skipped and not sent to OpenSearch.
The bug is in the block ([A-Z0-9-]+) from the pattern, related to the field ssl_cipher (documentation).
A lot of log entries uses the character _ instead of - in this field. When it happens, the line is skipped.
Expected Behavior
Not to skip log lines that use the character _ on the ssl_cipher field.
Current Behavior
Lines that use the character _ on the ssl_cipher field are skipped by the Log Parser.
Example: TLS_AES_128_GCM_SHA256
Reproduction Steps
Ingest any log file with the character _ in the ssl_cipher field.
Possible Solution
Add the _ character in the RegEx block. Like this: ([A-Z0-9-_]+)
I believe this is the correct file to make the change (line 165).
JoeShi
commented
8 months ago
Describe the bug
When deploying an ELB Service Log Pipeline, a Lambda Function named
xxxxxx-LogProcessorFnis deployed with the description: "Function to process and load ELB logs into OpenSearch"Inside this Lambda, there's the file
util/log_parser.pyIn the
ELBPython class, the methodparse(line 88) checks if a line from the log file matches a specific RegEx pattern. If it doesn't, the line is skipped and not sent to OpenSearch.The bug is in the block
([A-Z0-9-]+)from the pattern, related to the fieldssl_cipher(documentation).A lot of log entries uses the character
_instead of-in this field. When it happens, the line is skipped.Expected Behavior
Not to skip log lines that use the character
_on thessl_cipherfield.Current Behavior
Lines that use the character
_on thessl_cipherfield are skipped by the Log Parser. Example:TLS_AES_128_GCM_SHA256Reproduction Steps
Ingest any log file with the character
_in thessl_cipherfield.Possible Solution
Add the
_character in the RegEx block. Like this:([A-Z0-9-_]+)I believe this is the correct file to make the change (line 165).Additional Information/Context
No response
Solution Version
2.0.1
AWS Region. e.g., us-east-1
No response
Other information
No response