search

tpenakov / otaibe-quarkus-elasticsearch-example

Example project for elasti search high level client usage
MIT License
0 stars 2 forks source link

Build fails #1

Closed batlogs closed 4 years ago

batlogs commented 4 years ago

I did follow your instructions and not able to get the application to run. I keep getting "java.util.ServiceConfigurationError: org.eclipse.microprofile.config.spi.Converter: Provider io.quarkus.runtime.configuration.InetSocketAddressConverter not found" error.

No luck trying to run FruitResourceTests always fails with 405 error! not sure if that is related to the above error, I did skip tests to build the native image

tpenakov commented 4 years ago

I've changed the native build to go through Docker. Could you please try the following steps:

tpenakov commented 4 years ago

The instructions above are added to Readme.md file

batlogs commented 4 years ago

I am able to get the docker to run, but the http://localhost:11027 port (as mentioned on the blog) is not accessible. Connection times out.

batlogs commented 4 years ago

On further digging, I found the docker image never got built. It failed with "Error: Image build request failed with exit status 137".

tpenakov commented 4 years ago

Hi @batlogs,

Did you tried to build the image through the build script?

bash src/test/resources/shell/docker_build_and_run.sh

Could you please provide the output from the script?

batlogs commented 4 years ago

bash src/test/resources/shell/docker_build_and_run.sh

batlogs commented 4 years ago

Figured Error 137 maybe caused by low memory, I am running this on 16GB RAM Mac Pro. Hence I did make the below change to the script

mvn clean package -Pnative -Dquarkus.native.container-build=true -Dquarkus.profile=${PROFILE} && \
  docker build --network host -J-Xmx10G -J-Xms5G -f src/main/docker/Dockerfile.native -t ${IMAGE_NAME} . && \
  docker tag ${IMAGE_NAME}:latest ${IMAGE_NAME}:${PROFILE} && \
  docker run -i --rm -p ${PORT}:${PORT} \
    --name=${SERVICE_NAME} \
    --network host \
    ${IMAGE_NAME}:${PROFILE}

It's currently taking more than 8 mins to fail, when it fails it also brings down the elasticsearch instance

tpenakov commented 4 years ago

Hi @batlogs , Is it fail with the same error? Can you try to reduce the memory? e.g: -J-Xmx8g

batlogs commented 4 years ago

I did try with -J-Xmx8g and it too failed with Error 137. Hence I have increased it to 10

tpenakov commented 4 years ago

A few suggestions:

batlogs commented 4 years ago

It failed with -J-XMx10g & -J-Xms5G will try with 4G. If you have a prebuilt docker, I could test it. But I would eventually want to create my own image with my own domains. So, I'll need to get to the sweet spot of the configuration to succeed.

tpenakov commented 4 years ago

Hi @batlogs The image is triphon/otaibe-quarkus-elasticsearch-example It should have the two tags: latest and staging. They are the same.

tpenakov commented 4 years ago

Hi @batlogs , I think this might be your problem - by default Docker under mac is started with 1cpu/2048MB RAM.

First you should increase the memory there. Check this StackOverflow thread.

batlogs commented 4 years ago

Hi @batlogs , I think this might be your problem - by default Docker under mac is started with 1cpu/2048MB RAM.

First you should increase the memory there. Check this StackOverflow thread.

I am using docker desktop with the below settings

Screenshot 2020-06-01 at 9 55 00 AM
batlogs commented 4 years ago

Hi @batlogs The image is triphon/otaibe-quarkus-elasticsearch-example It should have the two tags: latest and staging. They are the same.

How do I run tests against this image?mvn package -D%test.service.http.host=http://localhost:11025 is giving a connection refused error with the below trace - the test is trying to connect to http://0.0.0.0:8081 is this correct?

mvn package -D%test.service.http.host=http://0.0.0.0:11025
[INFO] Scanning for projects...
[INFO] 
[INFO] --< org.otaibe.quarkus.elasticsearch.example:otaibe-quarkus-elasticsearch-example >--
[INFO] Building otaibe-quarkus-elasticsearch-example 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) @ otaibe-quarkus-elasticsearch-example ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.otaibe.quarkus.elasticsearch.example.web.controller.FruitResourceTest
2020-06-01 09:59:55,432 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init started
2020-06-01 09:59:55,873 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init completed
2020-06-01 09:59:55,873 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) init started
2020-06-01 09:59:55,873 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init started
2020-06-01 09:59:55,876 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init completed
2020-06-01 09:59:55,876 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) init completed
2020-06-01 09:59:55,876 INFO  [org.ota.qua.ela.exa.dao.FruitDaoImpl] (main) init started
2020-06-01 09:59:55,876 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (main) init started
2020-06-01 09:59:55,973 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [ml] on node [I_sYzF0eRJ6mF9Ui6acUig]
2020-06-01 09:59:55,973 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [ml] on node [I_sYzF0eRJ6mF9Ui6acUig]
2020-06-01 09:59:56,105 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (I/O dispatcher 11) index fruit exists=true
2020-06-01 09:59:56,105 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (I/O dispatcher 11) init completed
2020-06-01 09:59:56,131 INFO  [io.quarkus] (main) Quarkus 1.1.0.Final started in 1.602s. Listening on: http://0.0.0.0:8081
2020-06-01 09:59:56,131 INFO  [io.quarkus] (main) Profile test activated. 
2020-06-01 09:59:56,131 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy, resteasy-jackson]
2020-06-01 09:59:56,152 INFO  [org.ota.qua.ela.exa.web.con.FruitResourceTest] (main) restEndpointsTest start
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.301 s <<< FAILURE! - in org.otaibe.quarkus.elasticsearch.example.web.controller.FruitResourceTest
[ERROR] restEndpointsTest  Time elapsed: 0.712 s  <<< ERROR!
java.net.ConnectException: Connection refused (Connection refused)
    at org.otaibe.quarkus.elasticsearch.example.web.controller.FruitResourceTest.restEndpointsTest(FruitResourceTest.java:50)

[INFO] Running org.otaibe.quarkus.elasticsearch.example.service.FruitServiceTests
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
2020-06-01 09:59:57,548 INFO  [org.ota.qua.ela.exa.ser.FruitServiceTests] (main) saved result: {"id":"45b9a434-d78c-4814-bf03-153adfb4dc52","ext_ref_id":"ad8dd55d-d618-4c9f-ae43-2c007f3fc64b","name":"Apple","description":"Golden Apples from Bulgaria","version":1}
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.939 s - in org.otaibe.quarkus.elasticsearch.example.service.FruitServiceTests
2020-06-01 09:59:57,817 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) shutdown started
2020-06-01 09:59:57,817 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) shutdown started
2020-06-01 09:59:58,821 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) shutdown completed
2020-06-01 09:59:58,821 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) shutdown completed
2020-06-01 09:59:58,845 INFO  [io.quarkus] (main) Quarkus stopped in 1.029s
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   FruitResourceTest.restEndpointsTest:50 » Connect Connection refused (Connectio...
[INFO] 
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.110 s
[INFO] Finished at: 2020-06-01T09:59:59+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.1:test (default-test) on project otaibe-quarkus-elasticsearch-example: There are test failures.
[ERROR] 
[ERROR] Please refer to /Users/a257547/usr/a257547/trial/quarkus/otaibe-quarkus-elasticsearch-example/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
tpenakov commented 4 years ago

Hi @batlogs , On your Docker desktop - if you want to build the image by your self - you have to increase the memory there - for example to 8Gb and then to add -J-XMx4g to your build script.

In order to test the Docker Build - you have to start the triphon/otaibe-quarkus-elasticsearch-example or your own build - you can check the src/test/resources/shell/docker_build_and_run.sh script how to do it. It will start the image and will listen on 11025 port. Then you have to start the tests:

mvn package -D%test.service.http.host=http://localhost:11025

In that way he tests will hit the native docker build.

batlogs commented 4 years ago

`#If you want to test native docker build run docker image and uncomment this

%test.service.http.host=http://localhost:11025`

Should this be uncommented in application.properties before running the script, I have tired without uncommenting the above config, but the mvn package command fails with connection refused error.

Same happens when I uncomment.

But as a part of the build the tests do run before composing the image right?

In any case the mvn package -D%test.service.http.host=http://localhost:11025 is failing with connection refused error

Also which port to use for testing the application from maybe postman

P.S: The change in the ressources did help the image build

tpenakov commented 4 years ago

Hi @batlogs ,

Connection Refused error is because you are running the tests without to have an native build image running. 

mvn package -D%test.service.http.host=http://localhost:11025

The command above will run the tests against the running application.

Please follow the steps from the Readme.md file. It will:

I am not using postman. The tests are made in a way to run against different environments.

batlogs commented 4 years ago

I am able to generate the native image by running the bash src/test/resources/shell/docker_build_and_run.sh, when the this image is running, I tried to run the mvn package -D%test.service.http.host=http://localhost:11025 on another terminal which is failing.

If I have to curl for the FruitResource.save()endpoint, what would be the port to use?

tpenakov commented 4 years ago

The application by default is running on port 11025. So the Docker image started form src/test/resources/shell/docker_build_and_run.sh is running on that port too. This seems like connectivity issue between the Docker and Mac.

You can try to get even empty resource :

curl -v http://localhost:11025/fruits/any-id

and to check what the curl will say

batlogs commented 4 years ago

I am getting the below error when I try to run curl

curl -v http://localhost:11025/fruits/1
*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 11025 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connection failed
* connect to 127.0.0.1 port 11025 failed: Connection refused
* Failed to connect to localhost port 11025: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 11025: Connection refused
tpenakov commented 4 years ago

Lets try this:

If the tests went ok - you have to search for the problem somewhere between the MacOS - Docker connectivity.

batlogs commented 4 years ago

The below error is seen even when the dev mode is active.

mvn package -D%test.service.http.host=http://localhost:11025
[INFO] Scanning for projects...
[INFO] 
[INFO] --< org.otaibe.quarkus.elasticsearch.example:otaibe-quarkus-elasticsearch-example >--
[INFO] Building otaibe-quarkus-elasticsearch-example 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ otaibe-quarkus-elasticsearch-example ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) @ otaibe-quarkus-elasticsearch-example ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.otaibe.quarkus.elasticsearch.example.web.controller.FruitResourceTest
2020-06-01 22:22:57,716 INFO  [org.ota.qua.ela.exa.dao.FruitDaoImpl] (main) init started
2020-06-01 22:22:57,718 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (main) init started
2020-06-01 22:22:57,720 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init started
2020-06-01 22:22:57,993 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init completed
2020-06-01 22:22:58,061 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [transform] on node [YM_-zhDWT5yWTv-Xr3M9aA]
2020-06-01 22:22:58,061 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [remote_cluster_client] on node [YM_-zhDWT5yWTv-Xr3M9aA]
2020-06-01 22:22:58,061 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [ml] on node [YM_-zhDWT5yWTv-Xr3M9aA]
2020-06-01 22:22:58,156 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (I/O dispatcher 1) index fruit exists=false
2020-06-01 22:22:58,469 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (I/O dispatcher 1) CreateIndexResponse: org.elasticsearch.client.indices.CreateIndexResponse@5e682b3
2020-06-01 22:22:58,469 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (I/O dispatcher 1) index fruit created=true
2020-06-01 22:22:58,469 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (I/O dispatcher 1) init completed
2020-06-01 22:22:58,470 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) init started
2020-06-01 22:22:58,470 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init started
2020-06-01 22:22:58,473 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init completed
2020-06-01 22:22:58,473 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) init completed
2020-06-01 22:22:58,474 INFO  [io.quarkus] (main) Quarkus 1.1.0.Final started in 1.367s. Listening on: http://0.0.0.0:8081
2020-06-01 22:22:58,474 INFO  [io.quarkus] (main) Profile test activated. 
2020-06-01 22:22:58,475 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy, resteasy-jackson]
2020-06-01 22:22:58,484 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [transform] on node [YM_-zhDWT5yWTv-Xr3M9aA]
2020-06-01 22:22:58,484 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [remote_cluster_client] on node [YM_-zhDWT5yWTv-Xr3M9aA]
2020-06-01 22:22:58,484 WARN  [org.ela.cli.sni.ElasticsearchNodesSniffer] (es_rest_client_sniffer[T#1]) unknown role [ml] on node [YM_-zhDWT5yWTv-Xr3M9aA]
2020-06-01 22:22:58,501 INFO  [org.ota.qua.ela.exa.web.con.FruitResourceTest] (main) restEndpointsTest start
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.306 s <<< FAILURE! - in org.otaibe.quarkus.elasticsearch.example.web.controller.FruitResourceTest
[ERROR] restEndpointsTest  Time elapsed: 1.848 s  <<< FAILURE!
java.lang.AssertionError: 
1 expectation failed.
Expected status code <200> but was <500>.

    at org.otaibe.quarkus.elasticsearch.example.web.controller.FruitResourceTest.restEndpointsTest(FruitResourceTest.java:52)

[INFO] Running org.otaibe.quarkus.elasticsearch.example.service.FruitServiceTests
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
2020-06-01 22:23:01,365 INFO  [org.ota.qua.ela.exa.ser.FruitServiceTests] (main) saved result: {"id":"e28b8964-a83d-44ca-a6ef-28842a6ba334","ext_ref_id":"ad8dd55d-d618-4c9f-ae43-2c007f3fc64b","name":"Apple","description":"Golden Apples from Bulgaria","version":1}
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.207 s - in org.otaibe.quarkus.elasticsearch.example.service.FruitServiceTests
2020-06-01 22:23:01,567 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) shutdown started
2020-06-01 22:23:01,567 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) shutdown started
2020-06-01 22:23:02,571 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) shutdown completed
2020-06-01 22:23:02,571 INFO  [org.ota.qua.ela.exa.ser.ElasticsearchService] (main) shutdown completed
2020-06-01 22:23:02,597 INFO  [io.quarkus] (main) Quarkus stopped in 1.030s
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   FruitResourceTest.restEndpointsTest:52 1 expectation failed.
Expected status code <200> but was <500>.

[INFO] 
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10.296 s
[INFO] Finished at: 2020-06-01T22:23:03+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.1:test (default-test) on project otaibe-quarkus-elasticsearch-example: There are test failures.
[ERROR] 
[ERROR] Please refer to /Users/a257547/usr/a257547/trial/quarkus/otaibe-quarkus-elasticsearch-example/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
batlogs commented 4 years ago

The below is the error on the dev mode server

2020-06-01 22:43:43,737 ERROR [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (pool-3-thread-1) unable to save: java.net.ConnectException: Timeout connecting to [/172.17.0.2:9200]
    at org.apache.http.nio.pool.RouteSpecificPool.timeout(RouteSpecificPool.java:169)
    at org.apache.http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIOConnPool.java:628)
    at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.timeout(AbstractNIOConnPool.java:894)
    at org.apache.http.impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processTimeouts(DefaultConnectingIOReactor.java:210)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:155)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
    at java.lang.Thread.run(Thread.java:748)
tpenakov commented 4 years ago

Well - seems the same problem - Mac OS (your tests) can not connect to Docker (Elasticsearch run as docker container)

As far I can see in general you have a problems to connect your Docker containers all together and also your host OS can not communicate with them.

I am not sure is this can help - try to run the containers without --network host. Friend of mine told me that this might cause a problems under Mac and Windows

batlogs commented 4 years ago

The docker build doesn't go through if the --network host is removed. I can connect my elasticsearch instance running out of a docker with an external client running on my MacOS.

Just not sure where the application is unable to use the connection ports! I am usibg GraalVM instead of a JVM. Could that be a source of any connection issue?

tpenakov commented 4 years ago

The log attached from you above clearly says that you have a connectivity issues. You have to work in direction of resolving them.

BTW - this seems strange to me: 

service.elastic-search.hosts=localhost:11027

Why the exception is:

java.net.ConnectException: Timeout connecting to [/172.17.0.2:9200]

Did you using a different address in order to connect to elastic?

batlogs commented 4 years ago

I had changed the host in the Dockerfile.native while trying out some things. That's my bad!

After I removed that, I have modified the src/test/resources/shell/docker_build_and_run.sh like below, 

#!/bin/sh
set -x #echo on

# Staging
# bash src/test/resources/shell/docker_build_and_run.sh

export PROFILE=staging
export SERVICE_NAME=otaibe-quarkus-elasticsearch-example
export IMAGE_NAME=quarkus/${SERVICE_NAME}
export PORT=11025
export JAVA_HOME=/Library/Java/JavaVirtualMachines/graalvm-ce-java8-20.0.0/Contents/Home

mvn clean package -Pnative -Dquarkus.native.container-build=true -Dquarkus.profile=${PROFILE} && \
  docker build -f src/main/docker/Dockerfile.native -t ${IMAGE_NAME} . && \
  docker tag ${IMAGE_NAME}:latest ${IMAGE_NAME}:${PROFILE} && \
  docker run -i --rm -p ${PORT}:${PORT} \
    --name=${SERVICE_NAME} \
    ${IMAGE_NAME}:${PROFILE}

now the Build is successful but the quarkus init is failing with the below error

Successfully tagged quarkus/otaibe-quarkus-elasticsearch-example:latest
+ docker tag quarkus/otaibe-quarkus-elasticsearch-example:latest quarkus/otaibe-quarkus-elasticsearch-example:staging
+ docker run -i --rm -p 11025:11025 --name=otaibe-quarkus-elasticsearch-example quarkus/otaibe-quarkus-elasticsearch-example:staging
2020-06-02 16:34:45,011 INFO  [org.ota.qua.ela.exa.dao.FruitDaoImpl] (main) init started
2020-06-02 16:34:45,011 INFO  [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (main) init started
2020-06-02 16:34:45,011 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init started
WARNING: The sunec native library, required by the SunEC provider, could not be loaded. This library is usually shipped as part of the JDK and can be found under <JAVA_HOME>/jre/lib/<platform>/libsunec.so. It is loaded at run time via System.loadLibrary("sunec"), the first time services from SunEC are accessed. To use this provider's services the java.library.path system property needs to be set accordingly to point to a location that contains libsunec.so. Note that if java.library.path is not set it defaults to the current working directory.
2020-06-02 16:34:45,013 INFO  [org.ota.com.qua.ela.cli.ser.AbstractElasticsearchService] (main) init completed
2020-06-02 16:34:45,027 ERROR [org.ota.com.qua.ela.cli.dao.AbstractElasticsearchReactiveDaoImplementation] (pool-3-thread-1) unable to check for index: java.net.ConnectException: Connection refused
    at com.oracle.svm.core.posix.PosixJavaNIOSubstitutions$Util_sun_nio_ch_Net.handleSocketError(PosixJavaNIOSubstitutions.java:1315)
    at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java:3149)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
    at java.lang.Thread.run(Thread.java:748)
    at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:460)
    at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)

I don't believe that docker is not connecting through MacOS, there are other applications that I run on docker with no connectivity issue on this machine.

tpenakov commented 4 years ago 
java.net.ConnectException: Connection refused

Seems like connectivity issue for me. As far I understand here, now you are trying to connect the Elasticsearch sample application, which now is started as Docker container and Elasticsearch, which is started as Docker container too. Right? So now you have a problem connecting two docker containers to each other. In the original version of the src/test/resources/shell/docker_build_and_run.sh script the Elasticsearch sample application is started with --network host option which will use the Host OS network and that's why it can connect to the Elasticsearch by using this connection string:

service.elastic-search.hosts=localhost:11027

Starting it without this option will make Docker to try to connect to the address in the same Docker instance, which obviously will lead to the error above.

batlogs commented 4 years ago 
<dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-elasticsearch-rest-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>7.4.0</version>
        </dependency>

You use these libraries to connect with Elasticsearch right? Also, is there specific components that are used from below to make connections with Elasticsearch

<dependency>
            <groupId>com.github.tpenakov.otaibe-commons-quarkus</groupId>
            <artifactId>otaibe-commons-quarkus-core</artifactId>
            <version>${otaibe-commons-quarkus.version}</version>
        </dependency>
        <dependency>
            <groupId>com.github.tpenakov.otaibe-commons-quarkus</groupId>
            <artifactId>otaibe-commons-quarkus-elasticsearch</artifactId>
            <version>${otaibe-commons-quarkus.version}</version>
        </dependency>
        <dependency>
            <groupId>com.github.tpenakov.otaibe-commons-quarkus</groupId>
            <artifactId>otaibe-commons-quarkus-rest</artifactId>
            <version>${otaibe-commons-quarkus.version}</version>
        </dependency>

I created a sample docker quarkus app to check connectivity issue, but it worked just fine! so wanted to enhance this sample to just connect to Elasticsearch and query.

tpenakov commented 4 years ago

As far I understand - in order Native Docker Application to work and to be able to connect to Elasticsearch you have to adjust this property:

service.elastic-search.hosts=localhost:11027

It is set in that way and will work ONLY if the Native Docker Application container is started with --network host option. However I am not sure is the --network host option is working under mac at all. Here are some resources for connectivity between Docker standalone containers: https://docs.docker.com/network/network-tutorial-standalone/ Also you can try to connect them via Docker Compose And here is how you can override the properties in Quarkus: https://quarkus.io/guides/config You can check "Overriding properties at runtime" and "Configuration Profiles" there.

One more option: We can try to resolve this together via Hangout session. My id there is triphon__AT__gmail.com

batlogs commented 4 years ago

Very Gracious of you! I am in middle of somethings, I will take you up on your offer soon. Appreciate you help thus far. I Will ping you before hand to setup such a session :)

tpenakov commented 4 years ago

Ok - please prepare a clean Docker Native Build (with no git changes) before that.

batlogs commented 4 years ago

In the meantime I have created another sample app to better understand the issue. I have heavily borrowed the infrastructure from your existing repository, but I am still facing the same issue with inter docker communication

https://github.com/batlogs/quarkus-elastic-test

Please review it and advice

batlogs commented 4 years ago

Hi,

From what I understand on Quarkus devs, Elasticsearch plugin is still WIP. The elasticsearch-rest-high-level-client is a part of the Hibernate Integration with Elasticsearch. I was able to connect to ES with this Hibernate Integration. The downside of it is the I needed to use DB layer to persist my PanacheEntity.

Consecutively Quarkus folks seem to be working on Elasticsearch Client!

Just curious to know how you got this integration going? if it works well, maybe you can submit a PR for your current code and they review it (My humble suggestion)!

tpenakov commented 4 years ago

Actually the elasticsearch-rest-high-level-client works out of the box with quarkus. What the Quarkus team will do is to adjust the configuration in a quarkus way. Currently I didn't have time to do such a development - too much things will need to be polished ... The implementation here include only the bare minimum in order to make elasticsearch-rest-high-level-client to suit my needs.