cmanaha / python-elasticsearch-logger

Python Elasticsearch handler for the standard python logging framework
Other
232 stars 115 forks source link

AWS Lambda sends log to Cloudwatch instead of AWS Elasticsearch #58

Closed ganga-sah closed 5 years ago

ganga-sah commented 5 years ago

I used the CMRESHandler call below in AWS lambda. I had to add a new param "aws_token" with change in source code to successfully connect to AWS Elasticsearch using handler.test_essource() below. In spite of successful connection test from AWS lambda to AWS Elasticsearch, log.info() line at the end below, doesn't send log to Elasticsearch but instead shows up in Cloudwatch logs as below. [INFO] 2019-01-07T17:25:26.365Z 3860c080-12a1-11e9-8bd1-356fbb9f2564 Logging something to Elasticsearch Please suggest any possible fix to direct log to Elasticsearch as expected from code below. PS: The same code (uses localhost/9200 and w/o aws* args) is running fine on my Macbook with local install of Elasticsearch where logs are getting sent to local Elasticsearch.

handler = CMRESHandler(hosts=[{'host': elasticsearch_host, 'port': elasticsearch_port}],
                       auth_type=CMRESHandler.AuthType.AWS_SIGNED_AUTH,
                       aws_access_key=credentials.access_key,
                       aws_secret_key=credentials.secret_key,
                       aws_region=region,
                       aws_token=credentials.token,
                       use_ssl = True,
                       verify_ssl=True,
                       es_index_name="alogging",
                       es_additional_fields={'App': 'MyAppName', 'Environment': 'Dev'})

print "handler: ", handler
log = logging.getLogger("Clearedin")
for h in log.handlers:
    log.removeHandler(h)
log.setLevel(logging.DEBUG)
log.addHandler(handler)
print "handler test: ", handler.test_es_source()
log.info("Logging something to Elasticsearch")
ganga-sah commented 5 years ago

Luckily this has started working after removing line "log.removeHandler(h)" in above. Closing the issue.