Snowstorm on Google Kubernetes Engine

[svj23]

I was able to run it locally using latest jar file (3.0.3) and elastic search 6.5.4. Also created docker images with the help of docker-compose.yml. Pushed images of elastic search and snowstorm on GCP kubernetes. I was able to run elastic search on GCP after adding changes to increase max virtual memory. But snowstorm workload is not running, It shows error for loading elastic search.

Error - 2019-06-03 12:28:07.374 ERROR 1 --- [ main] .d.e.r.s.AbstractElasticsearchRepository : failed to load elasticsearch nodes : UncategorizedExecutionException[Failed execution]; nested: ExecutionException[java.net.ConnectException: Connection refused]; nested: ConnectException[Connection refused];; java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused

The elastic search is running on its own IP (load balancer) with port 80 though target port is mentioned 9200. I tried to change port to 9200 but then it goes to unhealthy state, So currently elastic search is running with port=80 and target port=9200. I don't know how to link snowstorm with elasticsearch service running on GCP. Please provide some guidelines or document for deployment process. Thank You.!

[kaicode]

Hi @svj23, I noticed that no one in the community responded to your request. Maybe no one has any experience of doing this.

Were you able to get Snowstorm working on Google Kubernetes Engine? If you figured it out we would love to hear about it!

Kind regards, Kai

[svj23]

Hi @kaicode, Thank you for asking, unfortunately I was also unable to figure it out. Regards.

[kevinbayes]

We are successfully running snowstorm on GKE. Once I have a moment to write this up I can, but will be next month though.

[AlexNolasco]

We managed to run snowstorm on Google Cloud using Elastic version 6.8.9 along with docker.elastic.co/elasticsearch/elasticsearch 6.8.3

Only issue we came across was the max_terms_count which we increased to 1000000

Prior to importing the codes we ran

PUT _all/_settings 
{
    "index": {
        "max_terms_count": 1000000
    }
}

[kaicode]

Thanks for sharing your findings @AlexNolasco.

Only issue we came across was the max_terms_count which we increased to 1000000 Yes, that can be used to avoid warnings from Elasticsearch.

In the next major version of Elasticsearch (7) requests will fail if the terms count is above the configured max, we will be looking into ways of reducing the number of terms or provide guidance on configuration as part of the upgrade work planned later this year.

[kaicode]

Closing due to inactivity. Thank you for your notes @AlexNolasco, people will still be able to find this information in the closed issues section or issue search.

[ivank]

@kaicode is it possible to include a k8s manifests that can work on EKS / GKE ? Since the my team is currently having trouble setting it up on GKE. You have this explanation of course but a custom manifest would be more helpful. > Depending of your kubernetes cluster you might want to change the location of where the elasticsearch data will be stored (actually HostPath storage model)

Something that sets "volumeClaimTemplates:" instead maybe? We're happy to contribute one, once we get it working but I was just wondering if you have something yourself to set us on our way.

[kaicode]

Hi @ivank. I have zero experience with K8s so am not able to help with this. It's not used in SNOMED International. Feel free to use this repository to create a new issue about this to get the attention of the community in case someone has solved this. I hope you get it working! We would be very happy to receive a contribution. Best of luck.

[ivank]

We failed to implement it ourselves, and opted just to use elastic cloud for a hosted elastic.

That way things obviously worked out quite nicely (though pricy)

[kaicode]

Thank you for letting us know! We have also moved to elastic.co for our production instances.

[charlmex]

Hello has any one deployed the snowmed browser in k8s successfuly? I am having some issues, the browser page loads, however does not seem to load correctly as in the case of Docker deployment.

• All searches returns concept not found
• Error while retrieving data from server...
• The release and version button at the top right are not even responsive

PS: the elsasticSearch, snowstorm and browser are all deployed in thesame cluster and namespace.

[kaicode]

Did the RF2 file import complete successfully? There should be a Snowstorm log message like this when the RF2 import completes:

11:55:24.793 [main] INFO  o.s.s.c.rf2.rf2import.ImportService - Completed RF2 SNAPSHOT import on branch MAIN in X seconds. ID 5b656f92-a7a9-4f2c-91ef-f46c850a5741

[charlmex]

yes it imported succefully. see log:

2024-11-14T13:42:40.659Z INFO 1 --- [pool-3-thread-1] o.s.s.core.rf2.rf2import.ImportService : Completed RF2 SNAPSHOT import on branch MAIN in 2808 seconds. ID acfe6fd3-681e-4d3b-8295-c804931c2cb2

[kaicode]

Are you able to access the Snowstorm API? By default the browser attempts to access Snowstorm using a URL relative to the browser /snowstorm/snomed-ct/codesystems So for example if the browser is at http://localhost:8080/ then it will try http://localhost:8080/snowstorm/snomed-ct/codesystems

This is because SNOMED International use the /snowstorm/snomed-ct context path for Snowstorm.

The fastest endpoint for testing is the version endpoint, for example http://localhost:8080/snowstorm/snomed-ct/version or using the default context http://localhost:8080/version

By default Snowstorm is not deployed under "snowstorm/snomed-ct". It can be configured to have this webapp path using a startup parameter like this:

--server.servlet.context-path=/snowstorm/snomed-ct

[charlmex]

Thank you so much, this fixed it.