search

awslabs / amazon-kinesis-agent

Continuously monitors a set of log files and sends new data to the Amazon Kinesis Stream and Amazon Kinesis Firehose in near-real-time.
Other
358 stars 222 forks source link

Kinesis Agent not starting : AccessDeniedException: /var/run/aws-kinesis-agent #39

Open deepakagra opened 8 years ago

deepakagra commented 8 years ago

I have installed agent on Ubuntu 14.04... It was working fine before ; but then it stopped suddenly and now getting following error while starting !

sudo service aws-kinesis-agent start

2016-07-10 18:46:40.373+0530 prod-api-ip-10-0-0-48 (main) com.amazon.kinesis.streaming.agent.Agent INFO Reading configuration from file: /etc/aws-kinesis/agent.json 2016-07-10 18:46:40.404+0530 prod-api-ip-10-0-0-48 (main) com.amazon.kinesis.streaming.agent.Agent INFO null: Agent will use up to 96 threads for sending data. 2016-07-10 18:46:40.411+0530 prod-api-ip-10-0-0-48 (main) com.amazon.kinesis.streaming.agent.Agent ERROR Unhandled error. java.lang.RuntimeException: Failed to create or connect to the checkpoint database. at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.connect(Unknown Source) at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.(Unknown Source) at com.amazon.kinesis.streaming.agent.Agent.(Unknown Source) at com.amazon.kinesis.streaming.agent.Agent.main(Unknown Source) Caused by: java.nio.file.AccessDeniedException: /var/run/aws-kinesis-agent at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) at java.nio.file.Files.createDirectory(Files.java:674) at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) at java.nio.file.Files.createDirectories(Files.java:767)

chaochenq commented 8 years ago

Hello,

This seems that you don't grant aws-kinesis-agent-user x+w access to /var/run/aws-kinesis-agent/. So the way agent work is that it will have a dedicated user (aws-kinesis-agent-user) that has limited access to your system while only focuses on tailing files and sending records.

By installing the agent when you run the setup script, it actually sets aws-kinesis-agent-user as the owner of /var/run/aws-kinesis-agent so that it gets the permission. Do you know what happened before it stopped working? Any changes to the permission? Or server reboot or things like that?

Related issue might already be tracked here:

https://github.com/awslabs/amazon-kinesis-agent/issues/36 https://github.com/awslabs/amazon-kinesis-agent/pull/38

urjitbhatia commented 8 years ago

@deepakagra use a custom location for the agent checkpointing. Ubuntu will drop permissions from /var/run on reboot etc.

In your agent config:

"checkpointFile": "/opt/aws-kinesis-agent/run/checkpoints"

Where you can setup the correct permissions on the /opt/aws-kinesis-agent/run directory in your AMI.

Basically, chmod ugo+rwx /opt/aws-kinesis-agent/run