Is it possible that kinesis agent send to the wrong region?

[kontorovskaya]

Hi,

We send records from the different source files to the different regions KinesisStream with the same name. And sometimes record is sent to the wrong region. Is it possible that the problem is in the configuration?

{ "Sources": [ { "Id": "JsonLogSource", "SourceType": "DirectorySource", "RecordParser": "SingleLineJson", "Directory": "C:\\us-west-1", "FileNameFilter": "*.log", "InitialPosition": 0 }, { "Id": "JsonLogSource2", "SourceType": "DirectorySource", "RecordParser": "SingleLineJson", "Directory": "C:\\eu-west-2", "FileNameFilter": "*.log", "InitialPosition": 0 }, { "Id": "JsonLogSource3", "SourceType": "DirectorySource", "RecordParser": "SingleLineJson", "Directory": "C:\\eu-central-1", "FileNameFilter": "*.log", "InitialPosition": 0 } ], "Sinks": [ { "Id": "US1FirehoseLogStream", "SinkType": "KinesisStream", "StreamName": "kstream", "Region": "us-west-1", "AccessKey": "...", "SecretKey": "..." }, { "Id": "EU1FirehoseLogStream", "SinkType": "KinesisStream", "StreamName": "kstream", "Region": "eu-west-2", "AccessKey": "...", "SecretKey": "..." }, { "Id": "EU2FirehoseLogStream", "SinkType": "KinesisStream", "StreamName": "kstream", "Region": "eu-central-1", "AccessKey": "...", "SecretKey": "..." } ], "Pipes": [ { "Id": "JsonLogSourceToFirehoseLogStream", "SourceRef": "JsonLogSource", "SinkRef": "US1FirehoseLogStream" }, { "Id": "JsonLogSourceToFirehoseLogStream2", "SourceRef": "JsonLogSource2", "SinkRef": "EU1FirehoseLogStream" }, { "Id": "JsonLogSourceToFirehoseLogStream3", "SourceRef": "JsonLogSource3", "SinkRef": "EU2FirehoseLogStream" } ] }

[dhhoang]

@kontorovskaya, I do see a couple of problems in the configuration

• Directory setting with backward-slash is not escaped (should be C:\\us-west-1 instead of C:\us-west-1)
• FileNameFilter should be wild card or the log file won't get picked up (*.log instead of .log)

Besides, I can't tell how the agent is sending to the wrong region without more specific information about what logs/region you expect to send to. I see that the JsonLogSource2 is reading from C:\eu-west-2 folder but you're sending it to EU1FirehoseLogStream. Is this what you expect?

[kontorovskaya]

Directory setting with backward-slash is not escaped (should be C:\us-west-1 instead of C:\us-west-1) FileNameFilter should be wild card or the log file won't get picked up (*.log instead of .log)

Sorry, that was my mistake with escaping Github markdown formatting.

Besides, I can't tell how the agent is sending to the wrong region without more specific information about what logs/region you expect to send to.

For example, I can see records from the C:\eu-central-1\cnem.log in both eu-central-1 and us-west-1. My Windows service write records to the C:\eu-central-1\cnem.log file with log4net library, also once per hour old cnem.log is archivated and new fresh cnem.log is created.