search

intersystems-community / irisdemo-demo-htap

Shows how IRIS can ingest thousands or millions of records per second while allowing for queries on the same cluster. Works on a single IRIS instance or on an IRIS cluster on the cloud.
MIT License
10 stars 10 forks source link

ingest-worker1 Container is unhealthy. #2

Closed eduard93 closed 4 years ago

eduard93 commented 4 years ago

docker-compose up fails with: 

ERROR: for htapui  Container "54ade6ad7efb" is unhealthy.
ERROR: Encountered errors while bringing up the project.

Log:

root@docker:/InterSystems/htap# docker-compose up
Creating network "htap_default" with the default driver
Creating htapirisdb ... done
Creating htapmaster ... done

ERROR: for ingest-worker1  Container "54ade6ad7efb" is unhealthy.

ERROR: for query-worker1  Container "54ade6ad7efb" is unhealthy.

ERROR: for htapui  Container "54ade6ad7efb" is unhealthy.
ERROR: Encountered errors while bringing up the project.
root@docker:/InterSystems/htap# docker ps -a
CONTAINER ID        IMAGE                                                  COMMAND                  CREATED              STATUS                       PORTS                                                                                                           NAMES
54ade6ad7efb        intersystemsdc/irisdemo-demo-htap:master-version-1.0   "/app/run.sh"            About a minute ago   Up About a minute (healthy)   0.0.0.0:10002->8080/tcp                                                                                         htapmaster
ea8de8f770bc        intersystemsdc/irisdemo-base-irisdb-community:latest   "/iris-main"             About a minute ago   Up About a minute (healthy)   0.0.0.0:10001->52773/tcp                                                                                        htapirisdb
root@docker:/InterSystems/htap# docker logs 54ade6ad7efb

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.4.RELEASE)

2019-10-29 16:28:29.454  INFO 8 --- [           main] com.irisdemo.htap.App                    : Starting App v1.0.0 on htapmaster with PID 8 (/app/app.jar started by root in /app)
2019-10-29 16:28:29.480  INFO 8 --- [           main] com.irisdemo.htap.App                    : No active profile set, falling back to default profiles: default
2019-10-29 16:28:54.659  INFO 8 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2019-10-29 16:28:55.619  INFO 8 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2019-10-29 16:28:55.620  INFO 8 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.17]
2019-10-29 16:28:56.718  INFO 8 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2019-10-29 16:28:56.726  INFO 8 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 25351 ms
2019-10-29 16:28:57.354  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Read QUERY statement from file TABLE_SELECT.sql: Select
        account_id,
        brokerageaccountnum,
        org,
        status,
        tradingflag,
        entityaccountnum,
        clientaccountnum,
        active_date,
        topaccountnum,
        repteamno,
        repteamname,
        office_name,
        region,
        basecurr,
        createdby,
        createdts,
        group_id,
        load_version_no
from SpeedTest.Account

2019-10-29 16:28:57.462  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Read QUERY statement from file TABLE_SELECT_ROW.sql: Select
        account_id,
        brokerageaccountnum,
        org,
        status,
        tradingflag,
        entityaccountnum,
        clientaccountnum,
        active_date,
        topaccountnum,
        repteamno,
        repteamname,
        office_name,
        region,
        basecurr,
        createdby,
        createdts,
        group_id,
        load_version_no
from SpeedTest.Account
where account_id=?

2019-10-29 16:28:57.463  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Read INSERT statement from file TABLE_INSERT.sql:insert into SpeedTest.Account
(
        account_id,
        brokerageaccountnum,
        org,
        status,
        tradingflag,
        entityaccountnum,
        clientaccountnum,
        active_date,
        topaccountnum,
        repteamno,
        repteamname,
        office_name,
        region,
        basecurr,
        createdby,
        createdts,
        group_id,
        load_version_no
)
values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)

2019-10-29 16:28:57.547  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting CONSUMER_THREADS_PER_WORKER= 1
2019-10-29 16:28:57.547  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting CONSUMER_TIME_BETWEEN_QUERIES_IN_MILLIS = 10
2019-10-29 16:28:57.548  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting INGESTION_THREADS_PER_WORKER = 2
2019-10-29 16:28:57.548  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting START_CONSUMERS = true
2019-10-29 16:28:57.549  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting DISABLE_JOURNAL_FOR_DROP_TABLE = true
2019-10-29 16:28:57.549  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting DISABLE_JOURNAL_FOR_TRUNCATE_TABLE = true
2019-10-29 16:28:57.549  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting INGESTION_JDBC_URL = jdbc:IRIS://htapirisdb:51773/USER
2019-10-29 16:28:57.549  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting INGESTION_JDBC_USERNAME = SuperUser
2019-10-29 16:28:57.550  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting INGESTION_JDBC_PASSWORD = sys
2019-10-29 16:28:57.550  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting INGESTION_BATCH_SIZE = 1000
2019-10-29 16:28:57.555  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting CONSUMER_JDBC_URL = jdbc:IRIS://htapirisdb:51773/USER
2019-10-29 16:28:57.556  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting CONSUMER_JDBC_USERNAME = SuperUser
2019-10-29 16:28:57.616  INFO 8 --- [           main] com.irisdemo.htap.config.Config          : Setting CONSUMER_JDBC_PASSWORD = sys
2019-10-29 16:28:57.628  INFO 8 --- [           main] com.irisdemo.htap.db.DatabaseService     : Creating data source for 'jdbc:IRIS://htapirisdb:51773/USER'...
2019-10-29 16:29:02.356  INFO 8 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2019-10-29 16:29:09.029  INFO 8 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'taskScheduler'
2019-10-29 16:29:10.044  INFO 8 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2019-10-29 16:29:10.117  INFO 8 --- [           main] com.irisdemo.htap.App                    : Started App in 50.357 seconds (JVM running for 57.157)
2019-10-29 16:29:15.318  INFO 8 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2019-10-29 16:29:15.326  INFO 8 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2019-10-29 16:29:15.719  INFO 8 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 392 ms

docker version:

root@docker:/InterSystems/htap# docker version
Client:
 Version:           18.06.1-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        e68fc7a
 Built:             Tue Aug 21 17:24:56 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.06.1-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       e68fc7a
  Built:            Tue Aug 21 17:23:21 2018
  OS/Arch:          linux/amd64
  Experimental:     false
root@docker:/InterSystems/htap# docker-compose version
docker-compose version 1.23.0-rc2, build 350a555e
docker-py version: 3.5.0
CPython version: 3.6.6
OpenSSL version: OpenSSL 1.1.0f  25 May 2017
eduard93 commented 4 years ago

Can't reproduce on Windows, but easily reproducible on Linux.

amirsamary commented 4 years ago

This is closed right? I just tried it on Linux and it works fine.

eduard93 commented 4 years ago

Doesn't work for me. Tried several different linux servers, same issue:

[root@zabbix htap]# docker-compose up
htapirisdb is up-to-date
Creating htapmaster ... done

ERROR: for htapui  Container "56d13b12d709" is unhealthy.

ERROR: for query-worker1  Container "56d13b12d709" is unhealthy.

ERROR: for ingest-worker1  Container "56d13b12d709" is unhealthy.
ERROR: Encountered errors while bringing up the project.
[root@zabbix htap]# docker version
Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:25:41 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea
  Built:            Wed Nov 13 07:24:18 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
[root@zabbix htap]# docker-compose version
docker-compose version 1.25.0, build 0a186604
docker-py version: 4.1.0
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019
[root@zabbix htap]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.6.1810 (Core)
Release:        7.6.1810
Codename:       Core
amirsamary commented 4 years ago

It seems you are not using the latest version. I can see that your master is version-1.0.

Did you git pull recently? The images should be version-2.0.5.

eduard93 commented 4 years ago

I've done

wget https://raw.githubusercontent.com/intersystems-community/irisdemo-demo-htap/master/docker-compose.yml
docker-compose up

Should I do something else beforehand?

amirsamary commented 4 years ago

Check to see if the docker-compose.yml file is using version 2.0.5 of images. If it is not, delete the docker-compose.yml file you have and run wget again to get the latest one.

eduard93 commented 4 years ago
amirsamary commented 4 years ago

That is very strange... The master is showing up on your docker ps as "healthy". Why is docker-compose thinking it is unhealthy? Anyway, I have upgraded docker-compose format from 2.4 to 3.7 and the depends_on clause doesn't support a condition clause anymore so that I had to remove it. Try it now and let me know.

eduard93 commented 4 years ago

Thank you, @amirsamary = works now!