search

duosecurity / duo_log_sync

MIT License
37 stars 28 forks source link

Send logs to AWS ELB #11

Open sriramkannan opened 4 years ago

sriramkannan commented 4 years ago

Not sure if this is already supported. But I am not able to send the data to my AWS ELB. The duosync.log says it's writing the logs but I don's see it getting written/sent.

Config: 

version: '1.0.0'
servers:
  - id: 'data-pipeline-dev'
    hostname: '***.elb.us-east-1.amazonaws.com'
    port: 8081
    protocol: 'TCP'

account:
  ikey: '***'
  skey: '***'
  hostname: 'api-***.duosecurity.com'
  endpoint_server_mappings:
    - endpoints: ['adminaction', 'auth']
      server: 'data-pipeline-dev'
  is_msp: False

duosync.log

2020-09-24 16:14:56 INFO     Starting DuoLogSync
2020-09-24 16:14:56 INFO     DuoLogSync: Opening connection to ***elb.us-east-1.amazonaws.com:8081
2020-09-24 16:14:56 INFO     duo_client Admin initialized for ikey: *****, host: api-**.duosecurity.com
2020-09-24 16:14:56 INFO     adminaction producer: fetching next logs after 120 seconds
2020-09-24 16:14:56 INFO     adminaction consumer: waiting for logs
2020-09-24 16:14:56 INFO     auth producer: fetching next logs after 120 seconds
2020-09-24 16:14:56 INFO     auth consumer: waiting for logs
2020-09-24 16:16:56 INFO     adminaction producer: fetching logs
2020-09-24 16:16:56 INFO     auth producer: fetching logs
2020-09-24 16:16:57 INFO     adminaction producer: adding 35 logs to the queue
2020-09-24 16:16:57 INFO     adminaction producer: added 35 logs to the queue
2020-09-24 16:16:57 INFO     adminaction producer: fetching next logs after 120 seconds
2020-09-24 16:16:57 INFO     adminaction consumer: received 35 logs from producer
2020-09-24 16:16:57 INFO     adminaction consumer: writing logs
2020-09-24 16:16:57 INFO     adminaction consumer: successfully wrote all logs
2020-09-24 16:16:57 INFO     adminaction consumer: saving latest log offset to a checkpointing file
2020-09-24 16:16:57 INFO     adminaction consumer: waiting for logs
2020-09-24 16:17:00 INFO     auth producer: adding 6 logs to the queue
2020-09-24 16:17:00 INFO     auth producer: added 6 logs to the queue
2020-09-24 16:17:00 INFO     auth producer: fetching next logs after 120 seconds
2020-09-24 16:17:00 INFO     auth consumer: received 6 logs from producer
2020-09-24 16:17:00 INFO     auth consumer: writing logs
2020-09-24 16:17:00 INFO     auth consumer: successfully wrote all logs
2020-09-24 16:17:00 INFO     auth consumer: saving latest log offset to a checkpointing file
2020-09-24 16:17:00 INFO     auth consumer: waiting for logs
2020-09-24 16:18:57 INFO     adminaction producer: fetching logs
2020-09-24 16:18:57 INFO     adminaction producer: no new logs available
2020-09-24 16:18:57 INFO     adminaction producer: fetching next logs after 120 seconds
2020-09-24 16:19:00 INFO     auth producer: fetching logs
2020-09-24 16:19:04 INFO     auth producer: adding 1 logs to the queue
2020-09-24 16:19:04 INFO     auth producer: added 1 logs to the queue
2020-09-24 16:19:04 INFO     auth producer: fetching next logs after 120 seconds
2020-09-24 16:19:04 INFO     auth consumer: received 1 logs from producer
2020-09-24 16:19:04 INFO     auth consumer: writing logs
2020-09-24 16:19:04 INFO     auth consumer: successfully wrote all logs
2020-09-24 16:19:04 INFO     auth consumer: saving latest log offset to a checkpointing file
2020-09-24 16:19:04 INFO     auth consumer: waiting for logs
2020-09-24 16:20:58 INFO     adminaction producer: fetching logs
2020-09-24 16:20:59 INFO     adminaction producer: no new logs available
2020-09-24 16:20:59 INFO     adminaction producer: fetching next logs after 120 seconds
2020-09-24 16:21:04 INFO     auth producer: fetching logs
2020-09-24 16:21:09 INFO     auth producer: adding 0 logs to the queue
2020-09-24 16:21:09 INFO     auth producer: added 0 logs to the queue
2020-09-24 16:21:09 INFO     auth producer: fetching next logs after 120 seconds
2020-09-24 16:21:09 INFO     auth consumer: received 0 logs from producer
2020-09-24 16:21:09 INFO     auth consumer: No logs to write
2020-09-24 16:21:09 INFO     auth consumer: waiting for logs
sriramkannan commented 4 years ago

Never mind. The ELB was listening on http. I changed it to tcp and it started working.

rohanbendre commented 4 years ago

Thanks for the update @sriramkannan. Glad it worked fine!