n3tsurge
commented
2 years ago This should be resolved with the latest install.sh on our quick start guide
Closed Korpsyb closed 2 years ago
n3tsurge
commented
2 years ago This should be resolved with the latest install.sh on our quick start guide
Describe the bug During the docker-compose startup reflexapi is failing to opensearch API. Opensearch container starts right up and
Docker-compose logs reflexapi :
reflex-api | During handling of the above exception, another exception occurred: reflex-api | reflex-api | Traceback (most recent call last): reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/bin/gunicorn", line 8, in
reflex-api | sys.exit(run())
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
reflex-api | WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/app/base.py", line 231, in run
reflex-api | super().run()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/app/base.py", line 72, in run
reflex-api | Arbiter(self).run()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/arbiter.py", line 58, in init
reflex-api | self.setup(app)
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/arbiter.py", line 118, in setup
reflex-api | self.app.wsgi()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
reflex-api | self.callable = self.load()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
reflex-api | return self.load_wsgiapp()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
reflex-api | return util.import_app(self.app_uri)
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/gunicorn/util.py", line 412, in import_app
reflex-api | app = app(*args, *kwargs)
reflex-api | File "/app/init.py", line 218, in create_app
reflex-api | if setup_complete() != True:
reflex-api | File "/app/init.py", line 137, in setup_complete
reflex-api | return es.indices.exists('reflex-settings')
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/opensearchpy/client/utils.py", line 177, in _wrapped
reflex-api | return func(args, params=params, headers=headers, **kwargs)
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/opensearchpy/client/indices.py", line 331, in exists
reflex-api | return self.transport.perform_request(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/opensearchpy/transport.py", line 405, in perform_request
reflex-api | raise e
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/opensearchpy/transport.py", line 368, in perform_request
reflex-api | status, headers_response, data = connection.perform_request(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/opensearchpy/connection/http_urllib3.py", line 264, in perform_request
reflex-api | raise ConnectionError("N/A", str(e), e)
reflex-api | opensearchpy.exceptions.ConnectionError: ConnectionError((<urllib3.connection.HTTPSConnection object at 0x7f380d373610>, 'Connection to opensearch timed out. (connect timeout=120)')) caused by: ConnectTimeoutError((<urllib3.connection.HTTPSConnection object at 0x7f380d373610>, 'Connection to opensearch timed out. (connect timeout=120)'))
reflex-api | 2022-09-13 22:33:57,339 - opensearch - WARNING - HEAD https://opensearch:9200/reflex-settings [status:N/A request:120.103s]
reflex-api | Traceback (most recent call last):
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
reflex-api | conn = connection.create_connection(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/util/connection.py", line 95, in create_connection
reflex-api | raise err
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection
reflex-api | sock.connect(sa)
reflex-api | socket.timeout: timed out
reflex-api |
reflex-api | During handling of the above exception, another exception occurred:
reflex-api |
reflex-api | Traceback (most recent call last):
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/opensearchpy/connection/http_urllib3.py", line 249, in perform_request
reflex-api | response = self.pool.urlopen(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
reflex-api | retries = retries.increment(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/util/retry.py", line 525, in increment
reflex-api | raise six.reraise(type(error), error, _stacktrace)
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/packages/six.py", line 770, in reraise
reflex-api | raise value
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
reflex-api | httplib_response = self._make_request(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 386, in _make_request
reflex-api | self._validate_conn(conn)
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
reflex-api | conn.connect()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connection.py", line 358, in connect
reflex-api | self.sock = conn = self._new_conn()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connection.py", line 179, in _new_conn
reflex-api | raise ConnectTimeoutError(
reflex-api | urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f3671803430>, 'Connection to opensearch timed out. (connect timeout=120)')
reflex-api | 2022-09-13 22:35:57,378 - opensearch - WARNING - HEAD https://opensearch:9200/reflex-settings [status:N/A request:120.035s]
reflex-api | Traceback (most recent call last):
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
reflex-api | conn = connection.create_connection(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/util/connection.py", line 95, in create_connection
reflex-api | raise err
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection
reflex-api | sock.connect(sa)
reflex-api | socket.timeout: timed out
reflex-api |
reflex-api | During handling of the above exception, another exception occurred:
reflex-api |
reflex-api | Traceback (most recent call last):
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/opensearchpy/connection/http_urllib3.py", line 249, in perform_request
reflex-api | response = self.pool.urlopen(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
reflex-api | retries = retries.increment(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/util/retry.py", line 525, in increment
reflex-api | raise six.reraise(type(error), error, _stacktrace)
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/packages/six.py", line 770, in reraise
reflex-api | raise value
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
reflex-api | httplib_response = self._make_request(
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 386, in _make_request
reflex-api | self._validate_conn(conn)
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
reflex-api | conn.connect()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connection.py", line 358, in connect
reflex-api | self.sock = conn = self._new_conn()
reflex-api | File "/root/.local/share/virtualenvs/-x-v5uFv0/lib/python3.8/site-packages/urllib3/connection.py", line 179, in _new_conn
reflex-api | raise ConnectTimeoutError(
reflex-api | urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f36718033d0>, 'Connection to opensearch timed out. (connect timeout=120)')
Docker Logs -f opensearch
`Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
This tool will be deprecated in the next major release of OpenSearch https://github.com/opensearch-project/security/issues/1755
OpenSearch Security Demo Installer Warning: Do not use on production or public reachable systems Basedir: /usr/share/opensearch OpenSearch install type: rpm/deb on NAME="Amazon Linux" OpenSearch config dir: /usr/share/opensearch/config OpenSearch config file: /usr/share/opensearch/config/opensearch.yml OpenSearch bin dir: /usr/share/opensearch/bin OpenSearch plugins dir: /usr/share/opensearch/plugins OpenSearch lib dir: /usr/share/opensearch/lib Detected OpenSearch Version: x-content-2.2.0 Detected OpenSearch Security Version: 2.2.0.0 /usr/share/opensearch/config/opensearch.yml seems to be already configured for Security. Quit. Enabling OpenSearch Security Plugin WARNING: A terminally deprecated method in java.lang.System has been called WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.2.0.jar) WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch WARNING: System::setSecurityManager will be removed in a future release WARNING: A terminally deprecated method in java.lang.System has been called WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.2.0.jar) WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security WARNING: System::setSecurityManager will be removed in a future release [2022-09-13T21:18:45,718][INFO ][o.o.n.Node ] [0b0ce347a9f9] version[2.2.0], pid[33], build[tar/b1017fa3b9a1c781d4f34ecee411e0cdf930a515/2022-08-09T02:27:25.256769336Z], OS[Linux/5.15.0-47-generic/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.4/17.0.4+8] [2022-09-13T21:18:45,724][INFO ][o.o.n.Node ] [0b0ce347a9f9] JVM home [/usr/share/opensearch/jdk], using bundled JDK [true] [2022-09-13T21:18:45,724][INFO ][o.o.n.Node ] [0b0ce347a9f9] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-9644572691103166221, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xms2g, -Xmx2g, -XX:MaxDirectMemorySize=1073741824, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true] [2022-09-13T21:18:47,892][WARN ][stderr ] [0b0ce347a9f9] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". [2022-09-13T21:18:47,892][WARN ][stderr ] [0b0ce347a9f9] SLF4J: Defaulting to no-operation (NOP) logger implementation [2022-09-13T21:18:47,892][WARN ][stderr ] [0b0ce347a9f9] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. [2022-09-13T21:18:47,921][INFO ][o.o.s.s.t.SSLConfig ] [0b0ce347a9f9] SSL dual mode is disabled [2022-09-13T21:18:47,922][INFO ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] OpenSearch Config path is /usr/share/opensearch/config [2022-09-13T21:18:48,443][INFO ][o.o.s.s.DefaultSecurityKeyStore] [0b0ce347a9f9] JVM supports TLSv1.3 [2022-09-13T21:18:48,446][INFO ][o.o.s.s.DefaultSecurityKeyStore] [0b0ce347a9f9] Config directory is /usr/share/opensearch/config/, from there the key- and truststore files are resolved relatively [2022-09-13T21:18:49,100][INFO ][o.o.s.s.DefaultSecurityKeyStore] [0b0ce347a9f9] TLS Transport Client Provider : JDK [2022-09-13T21:18:49,100][INFO ][o.o.s.s.DefaultSecurityKeyStore] [0b0ce347a9f9] TLS Transport Server Provider : JDK [2022-09-13T21:18:49,101][INFO ][o.o.s.s.DefaultSecurityKeyStore] [0b0ce347a9f9] TLS HTTP Provider : JDK [2022-09-13T21:18:49,101][INFO ][o.o.s.s.DefaultSecurityKeyStore] [0b0ce347a9f9] Enabled TLS protocols for transport layer : [TLSv1.3, TLSv1.2] [2022-09-13T21:18:49,102][INFO ][o.o.s.s.DefaultSecurityKeyStore] [0b0ce347a9f9] Enabled TLS protocols for HTTP layer : [TLSv1.3, TLSv1.2] [2022-09-13T21:18:49,608][INFO ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] Clustername: docker-cluster [2022-09-13T21:18:49,634][WARN ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] File /usr/share/opensearch/config/log4j2.properties has insecure file permissions (should be 0600) [2022-09-13T21:18:49,635][WARN ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] File /usr/share/opensearch/config/esnode.pem has insecure file permissions (should be 0600) [2022-09-13T21:18:49,635][WARN ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] File /usr/share/opensearch/config/esnode-key.pem has insecure file permissions (should be 0600) [2022-09-13T21:18:49,636][WARN ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] File /usr/share/opensearch/config/root-ca.pem has insecure file permissions (should be 0600) [2022-09-13T21:18:49,636][WARN ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] File /usr/share/opensearch/config/kirk.pem has insecure file permissions (should be 0600) [2022-09-13T21:18:49,636][WARN ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] File /usr/share/opensearch/config/kirk-key.pem has insecure file permissions (should be 0600) [2022-09-13T21:18:51,305][INFO ][o.o.p.c.PluginSettings ] [0b0ce347a9f9] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: true, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600 [2022-09-13T21:18:52,014][INFO ][o.o.i.r.ReindexPlugin ] [0b0ce347a9f9] ReindexPlugin reloadSPI called [2022-09-13T21:18:52,015][INFO ][o.o.i.r.ReindexPlugin ] [0b0ce347a9f9] Unable to find any implementation for RemoteReindexExtension [2022-09-13T21:18:52,090][INFO ][o.o.j.JobSchedulerPlugin ] [0b0ce347a9f9] Loaded scheduler extension: opendistro_anomaly_detector, index: .opendistro-anomaly-detector-jobs [2022-09-13T21:18:52,154][INFO ][o.o.j.JobSchedulerPlugin ] [0b0ce347a9f9] Loaded scheduler extension: reports-scheduler, index: .opendistro-reports-definitions [2022-09-13T21:18:52,157][INFO ][o.o.j.JobSchedulerPlugin ] [0b0ce347a9f9] Loaded scheduler extension: opendistro-index-management, index: .opendistro-ism-config [2022-09-13T21:18:52,170][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [aggs-matrix-stats] [2022-09-13T21:18:52,170][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [analysis-common] [2022-09-13T21:18:52,171][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [geo] [2022-09-13T21:18:52,171][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [ingest-common] [2022-09-13T21:18:52,171][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [ingest-geoip] [2022-09-13T21:18:52,171][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [ingest-user-agent] [2022-09-13T21:18:52,171][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [lang-expression] [2022-09-13T21:18:52,171][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [lang-mustache] [2022-09-13T21:18:52,172][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [lang-painless] [2022-09-13T21:18:52,172][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [mapper-extras] [2022-09-13T21:18:52,172][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [opensearch-dashboards] [2022-09-13T21:18:52,172][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [parent-join] [2022-09-13T21:18:52,172][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [percolator] [2022-09-13T21:18:52,173][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [rank-eval] [2022-09-13T21:18:52,173][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [reindex] [2022-09-13T21:18:52,173][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [repository-url] [2022-09-13T21:18:52,173][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [systemd] [2022-09-13T21:18:52,173][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded module [transport-netty4] [2022-09-13T21:18:52,174][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-alerting] [2022-09-13T21:18:52,175][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-anomaly-detection] [2022-09-13T21:18:52,175][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-asynchronous-search] [2022-09-13T21:18:52,175][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-cross-cluster-replication] [2022-09-13T21:18:52,175][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-geospatial] [2022-09-13T21:18:52,176][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-index-management] [2022-09-13T21:18:52,176][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-job-scheduler] [2022-09-13T21:18:52,176][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-knn] [2022-09-13T21:18:52,176][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-ml] [2022-09-13T21:18:52,176][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-notifications] [2022-09-13T21:18:52,177][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-notifications-core] [2022-09-13T21:18:52,177][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-observability] [2022-09-13T21:18:52,177][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-performance-analyzer] [2022-09-13T21:18:52,177][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-reports-scheduler] [2022-09-13T21:18:52,177][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-security] [2022-09-13T21:18:52,177][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [opensearch-sql] [2022-09-13T21:18:52,178][INFO ][o.o.p.PluginsService ] [0b0ce347a9f9] loaded plugin [repository-s3] [2022-09-13T21:18:52,214][INFO ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] Disabled https compression by default to mitigate BREACH attacks. You can enable it by setting 'http.compression: true' in opensearch.yml [2022-09-13T21:18:52,237][DEPRECATION][o.o.d.c.s.Settings ] [0b0ce347a9f9] [node.max_local_storage_nodes] setting was deprecated in OpenSearch and will be removed in a future release! See the breaking changes documentation for the next major version. [2022-09-13T21:18:52,258][INFO ][o.o.e.NodeEnvironment ] [0b0ce347a9f9] using [1] data paths, mounts [[/usr/share/opensearch/data (/dev/mapper/ubuntu--vg-ubuntu--lv)]], net usable_space [78gb], net total_space [97.8gb], types [ext4] [2022-09-13T21:18:52,259][INFO ][o.o.e.NodeEnvironment ] [0b0ce347a9f9] heap size [2gb], compressed ordinary object pointers [true] [2022-09-13T21:18:52,480][INFO ][o.o.n.Node ] [0b0ce347a9f9] node name [0b0ce347a9f9], node ID [E4-MUmUYQSuAbs0kIcbMpw], cluster name [docker-cluster], roles [ingest, remote_cluster_client, data, cluster_manager] [2022-09-13T21:18:58,528][WARN ][o.o.s.c.Salt ] [0b0ce347a9f9] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes [2022-09-13T21:18:58,608][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Message routing enabled: true [2022-09-13T21:18:58,681][INFO ][o.o.s.f.SecurityFilter ] [0b0ce347a9f9] indices are made immutable.
[2022-09-13T21:18:59,359][INFO ][o.o.a.b.ADCircuitBreakerService] [0b0ce347a9f9] Registered memory breaker.
[2022-09-13T21:19:00,055][INFO ][o.o.m.c.b.MLCircuitBreakerService] [0b0ce347a9f9] Registered ML memory breaker.
[2022-09-13T21:19:00,807][INFO ][o.o.t.NettyAllocator ] [0b0ce347a9f9] creating NettyAllocator with the following configs: [name=opensearch_configured, chunk_size=256kb, suggested_max_allocation_size=256kb, factors={opensearch.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=1mb}]
[2022-09-13T21:19:00,944][INFO ][o.o.d.DiscoveryModule ] [0b0ce347a9f9] using discovery type [single-node] and seed hosts providers [settings]
[2022-09-13T21:19:01,804][WARN ][o.o.g.DanglingIndicesState] [0b0ce347a9f9] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2022-09-13T21:19:02,597][INFO ][o.o.p.h.c.PerformanceAnalyzerConfigAction] [0b0ce347a9f9] PerformanceAnalyzer Enabled: false
[2022-09-13T21:19:02,638][INFO ][o.o.n.Node ] [0b0ce347a9f9] initialized
[2022-09-13T21:19:02,638][INFO ][o.o.n.Node ] [0b0ce347a9f9] starting ...
[2022-09-13T21:19:02,863][INFO ][o.o.t.TransportService ] [0b0ce347a9f9] publish_address {172.18.0.2:9300}, bound_addresses {0.0.0.0:9300}
[2022-09-13T21:19:03,181][INFO ][o.o.c.c.Coordinator ] [0b0ce347a9f9] cluster UUID [TK6rYYu7QT-8HUf3fnFoRg]
[2022-09-13T21:19:03,336][INFO ][o.o.c.s.MasterService ] [0b0ce347a9f9] elected-as-cluster-manager ([1] nodes joined)[{0b0ce347a9f9}{E4-MUmUYQSuAbs0kIcbMpw}{Gy3m6xZxQsaY9hOdN9LuBw}{172.18.0.2}{172.18.0.2:9300}{dimr}{shard_indexing_pressure_enabled=true} elect leader, _BECOME_CLUSTER_MANAGERTASK, _FINISHELECTION], term: 8, version: 40, delta: cluster-manager node changed {previous [], current [{0b0ce347a9f9}{E4-MUmUYQSuAbs0kIcbMpw}{Gy3m6xZxQsaY9hOdN9LuBw}{172.18.0.2}{172.18.0.2:9300}{dimr}{shard_indexing_pressure_enabled=true}]}
[2022-09-13T21:19:03,462][INFO ][o.o.c.s.ClusterApplierService] [0b0ce347a9f9] cluster-manager node changed {previous [], current [{0b0ce347a9f9}{E4-MUmUYQSuAbs0kIcbMpw}{Gy3m6xZxQsaY9hOdN9LuBw}{172.18.0.2}{172.18.0.2:9300}{dimr}{shard_indexing_pressure_enabled=true}]}, term: 8, version: 40, reason: Publication{term=8, version=40}
[2022-09-13T21:19:03,479][INFO ][o.o.a.c.ADClusterEventListener] [0b0ce347a9f9] Cluster is not recovered yet.
[2022-09-13T21:19:03,489][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [0b0ce347a9f9] Detected cluster change event for destination migration
[2022-09-13T21:19:03,540][INFO ][o.o.i.i.ManagedIndexCoordinator] [0b0ce347a9f9] Cache cluster manager node onClusterManager time: 1663103943539
[2022-09-13T21:19:03,548][WARN ][o.o.p.c.s.h.ConfigOverridesClusterSettingHandler] [0b0ce347a9f9] Config override setting update called with empty string. Ignoring.
[2022-09-13T21:19:03,573][INFO ][o.o.h.AbstractHttpServerTransport] [0b0ce347a9f9] publish_address {172.18.0.2:9200}, bound_addresses {0.0.0.0:9200}
[2022-09-13T21:19:03,574][INFO ][o.o.n.Node ] [0b0ce347a9f9] started
[2022-09-13T21:19:03,574][INFO ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] Node started
[2022-09-13T21:19:03,575][INFO ][o.o.s.c.ConfigurationRepository] [0b0ce347a9f9] Will attempt to create index .opendistro_security and default configs if they are absent
[2022-09-13T21:19:03,577][INFO ][o.o.s.OpenSearchSecurityPlugin] [0b0ce347a9f9] 0 OpenSearch Security modules loaded so far: []
[2022-09-13T21:19:03,576][INFO ][o.o.s.c.ConfigurationRepository] [0b0ce347a9f9] Background init thread started. Install default config?: true
[2022-09-13T21:19:03,647][INFO ][o.o.c.s.ClusterSettings ] [0b0ce347a9f9] updating [plugins.index_state_management.template_migration.control] from [0] to [-1]
[2022-09-13T21:19:03,653][INFO ][o.o.a.c.HashRing ] [0b0ce347a9f9] Node added: [E4-MUmUYQSuAbs0kIcbMpw]
[2022-09-13T21:19:03,658][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [0b0ce347a9f9] Detected cluster change event for destination migration
[2022-09-13T21:19:03,662][INFO ][o.o.a.c.HashRing ] [0b0ce347a9f9] Add data node to AD version hash ring: E4-MUmUYQSuAbs0kIcbMpw
[2022-09-13T21:19:03,666][INFO ][o.o.a.c.HashRing ] [0b0ce347a9f9] All nodes with known AD version: {E4-MUmUYQSuAbs0kIcbMpw=ADNodeInfo{version=2.2.0, isEligibleDataNode=true}}
[2022-09-13T21:19:03,666][INFO ][o.o.a.c.HashRing ] [0b0ce347a9f9] Rebuild AD hash ring for realtime AD with cooldown, nodeChangeEvents size 0
[2022-09-13T21:19:03,666][INFO ][o.o.a.c.HashRing ] [0b0ce347a9f9] Build AD version hash ring successfully
[2022-09-13T21:19:03,668][INFO ][o.o.a.c.ADDataMigrator ] [0b0ce347a9f9] Start migrating AD data
[2022-09-13T21:19:03,668][INFO ][o.o.a.c.ADDataMigrator ] [0b0ce347a9f9] AD job index doesn't exist, no need to migrate
[2022-09-13T21:19:03,668][INFO ][o.o.a.c.ADClusterEventListener] [0b0ce347a9f9] Init AD version hash ring successfully
[2022-09-13T21:19:03,718][INFO ][o.o.g.GatewayService ] [0b0ce347a9f9] recovered [1] indices into cluster_state
[2022-09-13T21:19:03,723][INFO ][o.o.s.c.ConfigurationRepository] [0b0ce347a9f9] Index .opendistro_security already exists
[2022-09-13T21:19:03,724][INFO ][o.o.s.c.ConfigurationRepository] [0b0ce347a9f9] Node started, try to initialize it. Wait for at least yellow cluster state....
[2022-09-13T21:19:04,350][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [0b0ce347a9f9] Detected cluster change event for destination migration
[2022-09-13T21:19:04,649][INFO ][o.o.c.r.a.AllocationService] [0b0ce347a9f9] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.opendistro_security][0]]]).
[2022-09-13T21:19:04,754][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [0b0ce347a9f9] Detected cluster change event for destination migration
[2022-09-13T21:19:04,764][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'config' with /usr/share/opensearch/config/opensearch-security/config.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2022-09-13T21:19:04,933][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id config, skipping update.
[2022-09-13T21:19:04,934][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'roles' with /usr/share/opensearch/config/opensearch-security/roles.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2022-09-13T21:19:04,957][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id roles, skipping update.
[2022-09-13T21:19:04,957][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'rolesmapping' with /usr/share/opensearch/config/opensearch-security/roles_mapping.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2022-09-13T21:19:04,980][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id rolesmapping, skipping update.
[2022-09-13T21:19:04,981][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'internalusers' with /usr/share/opensearch/config/opensearch-security/internal_users.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2022-09-13T21:19:05,008][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id internalusers, skipping update.
[2022-09-13T21:19:05,008][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'actiongroups' with /usr/share/opensearch/config/opensearch-security/action_groups.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2022-09-13T21:19:05,018][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id actiongroups, skipping update.
[2022-09-13T21:19:05,019][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'tenants' with /usr/share/opensearch/config/opensearch-security/tenants.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2022-09-13T21:19:05,025][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id tenants, skipping update.
[2022-09-13T21:19:05,025][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'nodesdn' with /usr/share/opensearch/config/opensearch-security/nodes_dn.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
[2022-09-13T21:19:05,031][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id nodesdn, skipping update.
[2022-09-13T21:19:05,032][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'whitelist' with /usr/share/opensearch/config/opensearch-security/whitelist.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
[2022-09-13T21:19:05,036][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id whitelist, skipping update.
[2022-09-13T21:19:05,037][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'allowlist' with /usr/share/opensearch/config/opensearch-security/allowlist.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=true
[2022-09-13T21:19:05,043][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id allowlist, skipping update.
[2022-09-13T21:19:05,044][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Will update 'audit' with /usr/share/opensearch/config/opensearch-security/audit.yml and populate it with empty doc if file missing and populateEmptyIfFileMissing=false
[2022-09-13T21:19:05,085][INFO ][o.o.s.s.ConfigHelper ] [0b0ce347a9f9] Index .opendistro_security already contains doc with id audit, skipping update.
[2022-09-13T21:19:05,411][INFO ][stdout ] [0b0ce347a9f9] [FINE] No subscribers registered for event class org.opensearch.security.securityconf.DynamicConfigFactory$NodesDnModelImpl
[2022-09-13T21:19:05,412][INFO ][stdout ] [0b0ce347a9f9] [FINE] No subscribers registered for event class org.greenrobot.eventbus.NoSubscriberEvent
[2022-09-13T21:19:05,413][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing on REST API is enabled.
[2022-09-13T21:19:05,413][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from REST API auditing.
[2022-09-13T21:19:05,413][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing on Transport API is enabled.
[2022-09-13T21:19:05,414][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] [AUTHENTICATED, GRANTED_PRIVILEGES] are excluded from Transport API auditing.
[2022-09-13T21:19:05,414][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing of request body is enabled.
[2022-09-13T21:19:05,414][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Bulk requests resolution is disabled during request auditing.
[2022-09-13T21:19:05,414][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Index resolution is enabled during request auditing.
[2022-09-13T21:19:05,414][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Sensitive headers auditing is enabled.
[2022-09-13T21:19:05,415][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing requests from kibanaserver users is disabled.
[2022-09-13T21:19:05,421][WARN ][o.o.s.a.r.AuditMessageRouter] [0b0ce347a9f9] No endpoint configured for categories [BAD_HEADERS, FAILED_LOGIN, MISSING_PRIVILEGES, GRANTED_PRIVILEGES, OPENDISTRO_SECURITY_INDEX_ATTEMPT, SSL_EXCEPTION, AUTHENTICATED, INDEX_EVENT, COMPLIANCE_DOC_READ, COMPLIANCE_DOC_WRITE, COMPLIANCE_EXTERNAL_CONFIG, COMPLIANCE_INTERNAL_CONFIG_READ, COMPLIANCE_INTERNAL_CONFIG_WRITE], using default endpoint
[2022-09-13T21:19:05,421][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing of external configuration is disabled.
[2022-09-13T21:19:05,421][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing of internal configuration is enabled.
[2022-09-13T21:19:05,421][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing only metadata information for read request is enabled.
[2022-09-13T21:19:05,421][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing will watch {} for read requests.
[2022-09-13T21:19:05,422][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing read operation requests from kibanaserver users is disabled.
[2022-09-13T21:19:05,422][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing only metadata information for write request is enabled.
[2022-09-13T21:19:05,422][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing diffs for write requests is disabled.
[2022-09-13T21:19:05,424][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing write operation requests from kibanaserver users is disabled.
[2022-09-13T21:19:05,424][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Auditing will watch for write requests.
[2022-09-13T21:19:05,424][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] .opendistro_security is used as internal security index.
[2022-09-13T21:19:05,425][INFO ][o.o.s.a.i.AuditLogImpl ] [0b0ce347a9f9] Internal index used for posting audit logs is null
[2022-09-13T21:19:05,426][INFO ][o.o.s.c.ConfigurationRepository] [0b0ce347a9f9] Hot-reloading of audit configuration is enabled
[2022-09-13T21:19:05,426][INFO ][o.o.s.c.ConfigurationRepository] [0b0ce347a9f9] Node '0b0ce347a9f9' initialized
[2022-09-13T21:20:03,544][INFO ][o.o.i.i.ManagedIndexCoordinator] [0b0ce347a9f9] Performing move cluster state metadata.
[2022-09-13T21:20:03,546][INFO ][o.o.i.i.MetadataService ] [0b0ce347a9f9] ISM config index not exist, so we cancel the metadata migration job.
[2022-09-13T21:21:03,545][INFO ][o.o.i.i.ManagedIndexCoordinator] [0b0ce347a9f9] Cancel background move metadata process.
[2022-09-13T21:21:03,547][INFO ][o.o.i.i.ManagedIndexCoordinator] [0b0ce347a9f9] Performing move cluster state metadata.
[2022-09-13T21:21:03,547][INFO ][o.o.i.i.MetadataService ] [0b0ce347a9f9] Move metadata has finished.
[2022-09-13T21:24:03,188][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T21:29:03,192][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T21:34:03,193][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T21:39:03,194][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T21:44:03,196][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T21:49:03,197][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T21:54:03,198][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T21:59:03,199][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:04:03,201][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:09:03,202][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:14:03,203][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:19:03,204][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:19:03,485][INFO ][o.o.a.t.CronTransportAction] [0b0ce347a9f9] Start running AD hourly cron.
[2022-09-13T22:19:03,505][INFO ][o.o.a.t.ADTaskManager ] [0b0ce347a9f9] Start to maintain running historical tasks
[2022-09-13T22:19:03,535][INFO ][o.o.a.c.HourlyCron ] [0b0ce347a9f9] Hourly maintenance succeeds
[2022-09-13T22:24:03,205][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:29:03,207][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:34:03,208][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep
[2022-09-13T22:39:03,210][INFO ][o.o.j.s.JobSweeper ] [0b0ce347a9f9] Running full sweep`
To Reproduce Steps to reproduce the behavior:
sudo bash -c "$(curl -fsSL https://raw.githubusercontent.com/reflexsoar/reflex-docs/main/quickstart/install.sh)"Desktop (please complete the following information): Ubuntu Server 20.04 Docker version 20.10.18, build b40c2f6 Docker Compose version v2.4.1
