splunk / splunk-aws-serverless-apps

Splunk AWS Serverless applications and Lambda blueprints
MIT License
55 stars 61 forks source link

Support messages with non-ascii characters in `splunk-cloudwatch-logs-processor` #17

Open cazlo opened 3 years ago

cazlo commented 3 years ago

What

When handling logs with non-ASCII information, the splunk-cloudwatch-logs-processor lambda blows up

Why

This is because it is trying to read the event string using the ASCII encoding https://github.com/splunk/splunk-aws-serverless-apps/blob/master/splunk-cloudwatch-logs-processor/index.js#L46

Quick Fix

A quick fix for this is to instead use utf-8 encoding when reading the AWSevent

Reproducing this issue

  1. base64 decode the sample event
  2. add some Chinese characters to one of the messages
  3. base64 encode sample event
  4. run integration test
  5. observe test failure
cazlo commented 3 years ago

It looks like we can guarantee messages will be in utf-8. From aws documentation:

Event messages must be UTF-8 encoded