Closed MarcoGeek closed 3 years ago
Same here, running on Ubuntu 18.04.5 in Docker.
It seems that the 4.0.0 tag is not present in Docker hub, which causes the pulls to fail. For now, you can work around the error by replacing all 4.0.0 tags to the latest versions present at https://hub.docker.com/u/opencti For example:
image: opencti/platform:4.0.0 --> image: opencti/platform:latest
image: opencti/worker:4.0.0 --> image: opencti/worker:latest
@SamuelHassine could you give us any updates on this? As I guess you are currently working on updating things to OpenCTI 4.0.0
Now I have this issue. Anything else I can try? Thanks in advance.
Pulling connector-history (opencti/connector-history:latest).. ERROR: manifest for opencti/connector-history:latest not found: manifest unknown: manifest unknown Pulling connector-history (opencti/connector-history:4.0.0)... ERROR: manifest for opencti/connector-history:4.0.0 not found: manifest unknown: manifest unknown Pulling connector-history (opencti/connector-history:3.2.1)... ERROR: manifest for opencti/connector-history:3.2.1 not found: manifest unknown: manifest unknown
The connector-history
seems to be new, so you could disable that be commenting out that part in the docker-compose
file. OpenCTI works fine without that connector.
OK Thanks. Now I have this. Any assistance is appreciated.
Creating docker_connector-export-file-stix_1 ... done
Creating docker_connector-export-file-csv_1 ... done
Creating docker_minio_1 ... done
Creating docker_rabbitmq_1 ... done
Creating docker_connector-opencti_1 ... done
Creating docker_connector-import-file-stix_1 ... done
Creating docker_grakn_1 ... done
Creating docker_redis_1 ... done
Creating docker_connector-import-file-pdf-observables_1 ... done
Creating docker_elasticsearch_1 ... done
Creating docker_opencti_1 ... done
Creating docker_worker_1 ... done
Creating docker_worker_2 ... done
Creating docker_worker_3 ... done
Attaching to docker_connector-export-file-csv_1, docker_connector-export-file-stix_1, docker_connector-opencti_1, docker_minio_1, docker_connector-import-file-stix_1, docker_rabbitmq_1, docker_elasticsearch_1, docker_redis_1, docker_connector-import-file-pdf-observables_1, docker_grakn_1, docker_opencti_1, docker_worker_2, docker_worker_3, docker_worker_1
connector-export-file-csv_1 | INFO:root:Listing Threat-Actors with filters null.
connector-export-file-stix_1 | INFO:root:Listing Threat-Actors with filters null.
connector-import-file-pdf-observables_1 | INFO:root:Listing Threat-Actors with filters null.
connector-import-file-stix_1 | INFO:root:Listing Threat-Actors with filters null.
connector-opencti_1 | INFO:root:Listing Threat-Actors with filters null.
elasticsearch_1 | {"type": "server", "timestamp": "2020-10-20T12:59:40,042Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "c19cd1c94f4b", "message": "version[7.9.1], pid[9], build[default/docker/083627f112ba94dffc1232e8b42b73492789ef91/2020-09-01T21:22:21.964974Z], OS[Linux/5.4.0-51-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/14.0.1/14.0.1+7]" }
elasticsearch_1 | {"type": "server", "timestamp": "2020-10-20T12:59:40,046Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "c19cd1c94f4b", "message": "JVM home [/usr/share/elasticsearch/jdk]" }
elasticsearch_1 | {"type": "server", "timestamp": "2020-10-20T12:59:40,047Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "c19cd1c94f4b", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-8616251272136070757, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" }
grakn_1 | ====================================================================================================
grakn1 | ____ ____ __ _____
grakn_1 | | || \ | || | / /| \ | | | || || _ \ | |
grakn_1 | | | ||| | | | | | | || | / / | \ | | | | ||| | | || | | | | |
grakn_1 | | | | || / | || || |/ / | | | | | | | | || |_| / | |__
grakn1 | | || || \ | || \ | | | | | | | || _ \ | |
grakn_1 | | || || | \ \ | | | || | \ \ | | \ | | || || || || | \ \ | |__
grakn_1 | |____||| _|| |||| \|| \| |____||||| \||
grakn_1 |
grakn_1 | THE KNOWLEDGE GRAPH
grakn_1 | ====================================================================================================
grakn_1 | Version: 1.8.3
minio_1 |
minio_1 | You are running an older version of MinIO released 3 months ago
minio_1 | Update: docker pull minio/minio:RELEASE.2020-10-18T21-54-12Z
minio_1 |
minio_1 |
minio_1 | Attempting encryption of all config, IAM users and policies on MinIO backend
minio_1 | Endpoint: http://172.18.0.5:9000 http://127.0.0.1:9000
minio_1 |
minio_1 | Browser Access:
minio_1 | http://172.18.0.5:9000 http://127.0.0.1:9000
minio_1 |
minio_1 | Object API (Amazon S3 compatible):
minio_1 | Go: https://docs.min.io/docs/golang-client-quickstart-guide
minio_1 | Java: https://docs.min.io/docs/java-client-quickstart-guide
minio_1 | Python: https://docs.min.io/docs/python-client-quickstart-guide
minio_1 | JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
minio_1 | .NET: https://docs.min.io/docs/dotnet-client-quickstart-guide
rabbitmq_1 | Configuring logger redirection
rabbitmq_1 | 2020-10-20 12:59:41.411 [debug] <0.287.0> Lager installed handler error_logger_lager_h into error_logger
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.290.0> Lager installed handler lager_forwarder_backend into error_logger_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.293.0> Lager installed handler lager_forwarder_backend into rabbit_log_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.296.0> Lager installed handler lager_forwarder_backend into rabbit_log_channel_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.299.0> Lager installed handler lager_forwarder_backend into rabbit_log_connection_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.302.0> Lager installed handler lager_forwarder_backend into rabbit_log_feature_flags_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.305.0> Lager installed handler lager_forwarder_backend into rabbit_log_federation_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.308.0> Lager installed handler lager_forwarder_backend into rabbit_log_ldap_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.314.0> Lager installed handler lager_forwarder_backend into rabbit_log_prelaunch_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.458 [debug] <0.311.0> Lager installed handler lager_forwarder_backend into rabbit_log_mirroring_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.459 [debug] <0.317.0> Lager installed handler lager_forwarder_backend into rabbit_log_queue_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.459 [debug] <0.320.0> Lager installed handler lager_forwarder_backend into rabbit_log_ra_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.459 [debug] <0.323.0> Lager installed handler lager_forwarder_backend into rabbit_log_shovel_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.459 [debug] <0.326.0> Lager installed handler lager_forwarder_backend into rabbit_log_upgrade_lager_event
rabbitmq_1 | 2020-10-20 12:59:41.483 [info] <0.44.0> Application lager started on node rabbit@17f5bcd4122a
opencti_1 | yarn run v1.19.1
opencti_1 | $ node --max_old_space_size=8192 build/index.js
opencti_1 | {"error":{"name":"DatabaseError","_error":{},"_showLocations":false,"_showPath":false,"time_thrown":"2020-10-20T12:59:40.082Z","data":{"reason":"Grakn seems down","category":"technical"},"internalData":{}},"level":"error","message":"[OPENCTI] GraphQL initialization fail","timestamp":"2020-10-20T12:59:40.083Z"}
opencti_1 | error Command failed with exit code 1.
opencti_1 | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
redis_1 | 1:C 20 Oct 2020 12:59:35.117 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 20 Oct 2020 12:59:35.117 # Redis version=6.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 20 Oct 2020 12:59:35.117 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 20 Oct 2020 12:59:35.118 Running mode=standalone, port=6379.
redis_1 | 1:M 20 Oct 2020 12:59:35.118 # Server initialized
redis_1 | 1:M 20 Oct 2020 12:59:35.118 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 20 Oct 2020 12:59:35.118 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 20 Oct 2020 12:59:35.118 * Ready to accept connections
worker_3 | INFO:root:Listing Threat-Actors with filters null.
worker_2 | INFO:root:Listing Threat-Actors with filters null.
worker_1 | INFO:root:Listing Threat-Actors with filters null.
connector-export-file-csv_1 | OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...
worker_1 | Traceback (most recent call last):
worker_1 | File "worker.py", line 300, in
I've also downgraded Grakn to make it work:
image: graknlabs/grakn:1.7.2
Thanks for your help. However, i get the same database error with grakn:1.7.2. Any other suggestion please. Do we need to change the port? opencti_1 | {"error":{"name":"DatabaseError","_er ror":{},"_showLocations":false,"_showPath":false,"time_thrown":"2020-10-20T15:26 :23.000Z","data":{"reason":"Grakn seems down","category":"technical"},"internalD ata":{}},"level":"error","message":"[OPENCTI] GraphQL initialization fail","time stamp":"2020-10-20T15:26:23.001Z"}
Status: Downloaded newer image for graknlabs/grakn:1.7.2
Starting docker_connector-export-file-csv_1 ... done
Starting docker_connector-opencti_1 ... done
Starting docker_redis_1 ... done
Starting docker_minio_1 ... done
Starting docker_connector-import-file-stix_1 ... done
Starting docker_rabbitmq_1 ... done
Starting docker_connector-export-file-stix_1 ... done
Starting docker_connector-import-file-pdf-observables_1 ... done
Recreating docker_grakn_1 ... done
Starting docker_elasticsearch_1 ... done
Recreating docker_opencti_1 ... done
Recreating docker_worker_1 ... done
Recreating docker_worker_2 ... done
Recreating docker_worker_3 ... done
Attaching to docker_connector-opencti_1, docker_connector-export-file-csv_1, docker_connector-import-file-stix_1, docker_redis_1, docker_connector-import-file-pdf-observables_1, docker_elasticsearch_1, docker_minio_1, docker_rabbitmq_1, docker_connector-export-file-stix_1, docker_grakn_1, docker_opencti_1, docker_worker_3, docker_worker_1, docker_worker_2
connector-export-file-csv_1 | INFO:root:Listing Threat-Actors with filters null.
connector-export-file-stix_1 | INFO:root:Listing Threat-Actors with filters null.
connector-import-file-pdf-observables_1 | INFO:root:Listing Threat-Actors with filters null.
connector-import-file-stix_1 | INFO:root:Listing Threat-Actors with filters null.
connector-opencti_1 | INFO:root:Listing Threat-Actors with filters null.
elasticsearch_1 | {"type": "server", "timestamp": "2020-10-20T15:26:22,675Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "c19cd1c94f4b", "message": "version[7.9.1], pid[6], build[default/docker/083627f112ba94dffc1232e8b42b73492789ef91/2020-09-01T21:22:21.964974Z], OS[Linux/5.4.0-51-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/14.0.1/14.0.1+7]" }
elasticsearch_1 | {"type": "server", "timestamp": "2020-10-20T15:26:22,680Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "c19cd1c94f4b", "message": "JVM home [/usr/share/elasticsearch/jdk]" }
elasticsearch_1 | {"type": "server", "timestamp": "2020-10-20T15:26:22,680Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "c19cd1c94f4b", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-8205326365964178451, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" }
grakn_1 | ====================================================================================================
grakn1 | ____ ____ __ _____
grakn_1 | | || \ | || | / /| \ | | | || || _ \ | |
grakn_1 | | | ||| | | | | | | || | / / | \ | | | | ||| | | || | | | | |
grakn_1 | | | | || / | || || |/ / | | | | | | | | || |_| / | |__
grakn1 | | || || \ | || \ | | | | | | | || _ \ | |
grakn_1 | | || || | \ \ | | | || | \ \ | | \ | | || || || || | \ \ | |__
grakn_1 | |____||| _|| |||| \|| \| |____||||| \||
grakn_1 |
grakn_1 | THE KNOWLEDGE GRAPH
grakn_1 | ====================================================================================================
grakn_1 | Version: 1.7.2
minio_1 |
minio_1 | You are running an older version of MinIO released 3 months ago
minio_1 | Update: docker pull minio/minio:RELEASE.2020-10-18T21-54-12Z
minio_1 |
minio_1 |
minio_1 | Endpoint: http://172.18.0.7:9000 http://127.0.0.1:9000
minio_1 |
minio_1 | Browser Access:
minio_1 | http://172.18.0.7:9000 http://127.0.0.1:9000
minio_1 |
minio_1 | Object API (Amazon S3 compatible):
minio_1 | Go: https://docs.min.io/docs/golang-client-quickstart-guide
minio_1 | Java: https://docs.min.io/docs/java-client-quickstart-guide
minio_1 | Python: https://docs.min.io/docs/python-client-quickstart-guide
minio_1 | JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
minio_1 | .NET: https://docs.min.io/docs/dotnet-client-quickstart-guide
opencti_1 | yarn run v1.19.1
opencti_1 | $ node --max_old_space_size=8192 build/index.js
opencti_1 | {"error":{"name":"DatabaseError","_error":{},"_showLocations":false,"_showPath":false,"time_thrown":"2020-10-20T15:26:23.000Z","data":{"reason":"Grakn seems down","category":"technical"},"internalData":{}},"level":"error","message":"[OPENCTI] GraphQL initialization fail","timestamp":"2020-10-20T15:26:23.001Z"}
opencti_1 | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
opencti_1 | error Command failed with exit code 1.
rabbitmq_1 | Configuring logger redirection
redis_1 | 1:C 20 Oct 2020 15:26:17.548 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 20 Oct 2020 15:26:17.548 # Redis version=6.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 20 Oct 2020 15:26:17.548 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 20 Oct 2020 15:26:17.549 Running mode=standalone, port=6379.
redis_1 | 1:M 20 Oct 2020 15:26:17.549 # Server initialized
redis_1 | 1:M 20 Oct 2020 15:26:17.549 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 20 Oct 2020 15:26:17.549 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 20 Oct 2020 15:26:17.550 Loading RDB produced by version 6.0.5
redis_1 | 1:M 20 Oct 2020 15:26:17.550 RDB age 8728 seconds
redis_1 | 1:M 20 Oct 2020 15:26:17.550 RDB memory usage when created 0.77 Mb
redis_1 | 1:M 20 Oct 2020 15:26:17.550 DB loaded from disk: 0.000 seconds
redis_1 | 1:M 20 Oct 2020 15:26:17.550 * Ready to accept connections
worker_1 | INFO:root:Listing Threat-Actors with filters null.
worker_1 | Traceback (most recent call last):
worker_1 | File "worker.py", line 300, in
Did you wait for a while to let it all startup nicely?
I did. I wait for a while but the error message is continue to appear. Is this normal? How long should I wait if i need to wait? Can I leave it for 30 minutes to see if that helps? Thanks
Initially It gave Database error and later it keeps repeating "OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration..."
No that is not normal, it should be ready within a few minutes.. What does your docker-compose.yml
file look like?
Here is my docker-compose.yml file. The thing is I can access the OpenCTI however it has some error. Also the API is not reachable is keep running on the ubuntu shell. any idea?
version: '3' services: grakn: image: graknlabs/grakn:1.8.3 ports:
connector-export-file-stix: image: opencti/connector-export-file-stix:latest environment:
You know what. The best way to install is using the OVA image. Its quick and work without any issue. Also it comes with portainer for management of the docker. I just deployed it and it looks fine. I am performing rest of the configuration. But I would like to thank you for all your response. You were assisting me with this. I spent one week to figure this out and to get landing page. Thanks So Much. Its time to explore it and take it to next level. I am closing the ticket. by the way on docker also I change it to latest >>rolling and it works with same error. Now no more exploring on deployment activity.
For information OpenCTI 4.0 is currently in development. So reserved to developers for now.
Thank You. Appreciate your update. just one last query my thehive is running in production. Just need ton know. how to send automatic notifications classified by severity to the Security Incident Response Platform (TheHive)
Hi @MarcoGeek . Can you create a specific feature request explaining your needs? Expected Workflow? Type of information? and all detail you have in mind. Thanks
Please do it in the openCTI main repository.
Ok Sure will do that. Appreciate your response.
Environment: Ubuntu 20.04 Server Running on Docker.
Error Reproduce:
root@openctiserver:/home/administrator/opt/docker# docker-compose --compatibility up Pulling opencti (opencti/platform:4.0.0)... ERROR: manifest for opencti/platform:4.0.0 not found: manifest unknown: manifest unknown root@openctiserver:/home/administrator/opt/docker#