allegro / embedded-elasticsearch

Tool that ease up creation of integration tests with Elasticsearch
Apache License 2.0
269 stars 81 forks source link

Performance issues with 6.3.0 #61

Open magaton opened 6 years ago

magaton commented 6 years ago

Hello, I am facing some performance issues with 6.3.0 only on CI server linux box, everything works well on my local mac (single IT is 5 times faster on mac than on linux box). The problem is definitely related to the ES version, since 6.2.3 works well on linux box too. I am using embedded-elasticsearch-2.7.0.

My EmbeddedES init looks like this

EmbeddedES embeddedElastic = EmbeddedElastic.builder().withElasticVersion("6.3.0") .withSetting(PopularProperties.HTTP_PORT, httpPort) .withSetting(PopularProperties.TRANSPORT_TCP_PORT, httpPort + 100) .withSetting("http.cors.allow-origin", "*").withSetting("http.cors.enabled", true) .withSetting("cluster.routing.allocation.disk.threshold_enabled", false) .withSetting("bootstrap.memory_lock", true).withEsJavaOpts("-Xmx1024m -Xmx1024m") .withStartTimeout(10, TimeUnit.MINUTES) .withInstallationDirectory( new File(System.getProperty("java.io.tmpdir"), "embedded-elasticsearch/server")) .withDownloadDirectory( new File(System.getProperty("java.io.tmpdir"), "/embedded-elasticsearch/download")) .withCleanInstallationDirectoryOnStop(false).build().start();

and I manage EmbeddedES lifecycle via @ClassRule.

The main symptom is that the time length to execute AllocationService and change cluster health status.

12:30:10.356 [EmbeddedElsHandler] INFO p.a.t.e.ElasticServer - [2018-07-20T12:30:10,355][INFO ][o.e.c.m.MetaDataCreateIndexService] [8nIAMw4] [dummies-2018.07.20-1] creating index, cause [api], templates [dummy-template], shards [1]/[0], mappings [document] 12:30:17.780 [EmbeddedElsHandler] INFO p.a.t.e.ElasticServer - [2018-07-20T12:30:17,780][INFO ][o.e.c.r.a.AllocationService] [8nIAMw4] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[dummies-2018.07.20-1][0]] ...]).

The consequence was that since I have configured socket-timeout: 10000 in high level rest client config, the tests were failing. If I increase socket-timeout to 30000 all tests pass, but it takes quite a long time to finish.

Any idea what could be the reason for such behaviour?

Thanks, Milan