search

awslabs / amazon-kinesis-agent

Continuously monitors a set of log files and sends new data to the Amazon Kinesis Stream and Amazon Kinesis Firehose in near-real-time.
Other
360 stars 223 forks source link

Agent failed to start after EC2(ubuntu) reboot #101

Open nemalipuri opened 7 years ago

nemalipuri commented 7 years ago

Checkpoints database file(/var/run/aws-kinesis-agent) is disappeared after EC2 reboot. Due to this agent started throwing below error.

2017-07-05 17:15:06.702-0600 localhost (main) com.amazon.kinesis.streaming.agent.Agent [INFO] Reading configuration from file: /etc/aws-kinesis/agent.json
2017-07-05 17:15:06.791-0600 localhost (main) com.amazon.kinesis.streaming.agent.processing.processors.AgentDataConverterChain [DEBUG] Using data converter chain: AgentDataConverterChain [com.amazon.kinesis.streaming.agent.processing.processors.LogToJSONDataConverter@6a983034 ]
2017-07-05 17:15:06.814-0600 localhost (main) com.amazon.kinesis.streaming.agent.Agent [INFO] null: Agent will use up to 15 threads for sending data.
2017-07-05 17:15:06.834-0600 localhost (main) com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore [DEBUG] Connecting to database /var/run/aws-kinesis-agent/checkpoints...
2017-07-05 17:15:06.844-0600 localhost (main) com.amazon.kinesis.streaming.agent.Agent [ERROR] Unhandled error.
java.lang.RuntimeException: Failed to create or connect to the checkpoint database.
     at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.connect(Unknown Source)
     at com.amazon.kinesis.streaming.agent.tailing.checkpoints.SQLiteFileCheckpointStore.<init>(Unknown Source)
     at com.amazon.kinesis.streaming.agent.Agent.<init>(Unknown Source)
     at com.amazon.kinesis.streaming.agent.Agent.main(Unknown Source)
Caused by: java.nio.file.AccessDeniedException: /var/run/aws-kinesis-agent
     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
     at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:383)
     at java.nio.file.Files.createDirectory(Files.java:630)
     at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
     at java.nio.file.Files.createDirectories(Files.java:720)
     ... 4 more
chaochenq commented 7 years ago

Seems like related to this

https://github.com/awslabs/amazon-kinesis-agent/issues/39

nemalipuri commented 7 years ago

thanks for quick response. I will use a custom location for the agent checkpointing.