search

aws-samples / siem-on-amazon-opensearch-service

A solution for collecting, correlating and visualizing multiple types of logs to help investigate security incidents.
MIT No Attribution
579 stars 191 forks source link

Error when parsing lambda log: error.message:invalid json file: Expecting value: line 1 column 1 (char 0) #318

Closed ghost closed 1 year ago

ghost commented 2 years ago

Currently, I'm sending Lambda log from Cloud Watch log -> Kinesis Firehose -> S3. From S3 I'm using siem to perform ETL in order to import data into OpenSearch.

All my Lambda function logs are outputted as JSON format. But the Lambda default logs are not.

START RequestId: d0ba05dc-8506-11e8-82ab-afe2adba36e5 Version: $LATEST  <<< This is Lambda default log
{"message" : "Hello world"}  <<< This is the log from my Lambda code
END RequestId: d0ba05dc-8506-11e8-82ab-afe2adba36e5 <<< This is Lambda default log

So after I ran siem ETL, the log on OpenSearch get the following error: error.message:invalid json file: Expecting value: line 1 column 1 (char 0) Which is the result of Lambda default log parsing, I think.

How can I skip the above error? Note: I took a look at user.ini and aws.ini but It seems there is no setting that relate to this problem.

Thanks & Regards.

nakajiak commented 1 year ago

Sorry for the very late reply. Does the issue still exist? Please reopen if it exists