search

IHTSDO / snowstorm

Scalable SNOMED CT Terminology Server using Elasticsearch
Other
204 stars 80 forks source link

More robust startup, attempt Elasticsearch connection multiple times. #177

Open kaicode opened 3 years ago

kaicode commented 3 years ago

New ticket following comments made by @fadam-mm in #167.

When Snowstorm starts if Elasticsearch is not already up and available the connection between them fails. Snowstorm logs the error but does not try to reconnect and the process does not exit..

If Elasticsearch is not yet available when Snowstorm starts it should continue to attempt to connect to Elasticsearch for at least a minute (configurable). If after that time it still can not connect the Snowstorm JVM process should exit with a unix error code.

fadam-mm commented 3 years ago

Thanks @kaicode !

fadam-mm commented 3 years ago

Hi @kaicode !

Any idea in which release this could be addressed? On our side, we'd like this enhancement before we upgrade. Alternatively, any suggestions/workarounds in the meantime that could get the app to exit if elasticsearch hadn't started yet?

Thanks again!

kaicode commented 3 years ago

Hi @fadam-mm, I've made this change on the develop branch. We already have another release going through testing at the moment but I'll aim to make another small release with this change after that.

In the meantime if you have the capability and capacity to build the develop branch and check that it works for you that would be helpful. See new config item elasticsearch.connection.attempts and comments.

kaicode commented 3 years ago

I've had to pull this out of the develop branch because it doesn't pick up the environment credentials in our AWS environment.

fadam-mm commented 3 years ago

Hi @kaicode, will this be able to make the next release? I see the commits above were re-added, is it still an option for us to merge and build off develop branch? Just want to make sure the AWS environment credentials issue you mentioned above is something specific to your test environment and not part of release code.

Thx!

fadam-mm commented 3 years ago

Happy New Year! Just looking for any status update on this ticket, based on that we can evaluate options on our side. Thanks snowstorm team 👍

kaicode commented 3 years ago

I would really like to release this feature but it completely breaks our environments and I've not had a chance to debug it. We are using environment variables for the AWS credentials but they are not picked up. This must be something to do with the order in which components within the Spring context are created. Any help with this is welcome. I can't make any promises when this will be completed.