search

awslabs / amazon-kinesis-agent

Continuously monitors a set of log files and sends new data to the Amazon Kinesis Stream and Amazon Kinesis Firehose in near-real-time.
Other
360 stars 223 forks source link

[ERROR] -Unable to execute HTTP request #122

Open Rajeshkumar123 opened 7 years ago

Rajeshkumar123 commented 7 years ago

Hi,

I have installed Kinesis agent in my RHEL server 7.2 . My server is beyond proxy . When I try to run the kinesis agent I am getting the below error. Is there any way to configure proxy setting in kinesis agent file.

Agent.json

{
  "awsAccessKeyId":"xxxxxxxx",
  "awsSecretAccessKey":"xxxxxxzxxxx",
  "cloudwatch.emitMetrics": true,
  "firehose.endpoint": "https://firehose.us-east-2.amazonaws.com",

  "flows": [
    {
      "filePattern": "/var/log/aws-kinesis-agent/aws-kinesis-agent.log*",
      "deliveryStream": "FireshosetoES"
    }
  ]
}

Error

com.amazon.kinesis.streaming.agent.tailing.AsyncPublisher [ERROR] AsyncPublisher[fh:FireshosetoES:/var/log/aws-kinesis-agent/aws-kinesis-agent.log*]:RecordBuffer(id=2,records=6,bytes=1988) Retriable send error (com.amazonaws.AmazonClientException: Unable to execute HTTP request: Connect to firehose.us-east-2.amazonaws.com:80 [firehose.us-east-2.amazonaws.com/xx.xxx.225] failed: connect timed out). Will retry.
 (FileTailer[fh:FireshosetoES:/var/log/aws-kinesis-agent/aws-kinesis-agent.log*].MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.tailing.FileTailer [INFO] FileTailer[fh:FireshosetoES:/var/log/aws-kinesis-agent/aws-kinesis-agent.log*]: Tailer Progress: Tailer has parsed 110 records (4211697 bytes), transformed 0 records, skipped 0 records, and has successfully sent 0 records to destination.
 (Agent.MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.Agent [INFO] Agent: Progress: 110 records parsed (4211697 bytes), and 0 records sent successfully to destinations. Uptime: 270059ms
com.amazon.kinesis.streaming.agent.tailing.AsyncPublisher [ERROR] AsyncPublisher[fh:FireshosetoES:/var/log/aws-kinesis-agent/aws-kinesis-agent.log*]:RecordBuffer(id=3,records=8,bytes=3170) Retriable send error (com.amazonaws.AmazonClientException: Unable to execute HTTP request: Connect to firehose.us-east-2.amazonaws.com:80 [firehose.us-east-2.amazonaws.com/xx.xxx.225] failed: connect timed out). Will com.amazon.kinesis.streaming.agent.tailing.AsyncPublisher [ERROR] AsyncPublisher[fh:FireshosetoES:/var/log/aws-kinesis-agent/aws-kinesis-agent.log*]:RecordBuffer(id=4,records=51,bytes=7260) Retriable send error (com.amazonaws.AmazonClientException: Unable to execute HTTP request: Connect to firehose.us-east-2.amazonaws.com:80 [firehose.us-east-2.amazonaws.com/xx.xxx.225] failed: connect timed out). Will retry.
(FileTailer[fh:FireshosetoES:/var/log/aws-kinesis-agent/aws-kinesis-agent.log*].MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.tailing.FileTailer [INFO] FileTailer[fh:FireshosetoES:/var/log/aws-kinesis-agent/aws-kinesis-agent.log*]: Tailer Progress: Tailer has parsed 114 records (4213415 bytes), transformed 0 records, skipped 0 records, and has successfully sent 0 records to destination.
 (Agent.MetricsEmitter RUNNING) com.amazon.kinesis.streaming.agent.Agent [INFO] Agent: Progress: 114 records parsed (4213415 bytes), and 0 records sent successfully to destinations. Uptime: 300059ms
martindariocernadas commented 6 years ago

Same problem! Some proxy configuration missing? How to fix it?

jagilbert27 commented 4 years ago

I got those errors because proxy.

Fixed by adding proxy config arguments to the startup script start-aws-kinesis-agent as described in Issue 61

In my case the suggested syntax didn't work: -Dhtttps.proxyHost=https://proxy.mydomain.com But this did work: -Dhtttps.proxyHost=proxy.mydomain.com