search

Graylog2 / graylog-docker

Official Graylog Docker image
https://hub.docker.com/r/graylog/graylog/
Apache License 2.0
361 stars 133 forks source link

Graylog frontend is not displayed after proxying #124

Closed offlinejke closed 3 years ago

offlinejke commented 3 years ago

The problem is that after I put graylog, elastic and mongo on the same internal docker network next to nginx to cheat dns, graylog stops displaying the frontend at the domain address. There seem to be no errors, but it seems to me that some components are not pulled from elastic. All other services in containers on which dns is assigned by analogy work fine. Proxy works out everywhere. No errors were found in the nginx log.

mongodb:
    image: mongo:3
    volumes:
      - mongo_data:/data/db
    networks:
      vpcbr:
        ipv4_address: 172.20.0.10

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.10
    volumes:
      - es_data:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    networks:
      vpcbr:
        ipv4_address: 172.20.0.11

  graylog:
    image: graylog/graylog:3.3.5
    volumes:
      - graylog_data:/usr/share/graylog/data
    environment:

      - GRAYLOG_PASSWORD_SECRET=2FLGyoUpYDhkVMSo89EsSxqF84DMxSnuZgpHFTqepBti671f2H4l8RQzNRWB51HGYVeLASVDlo3jYLkDxjLvoQPjE7uWnHKh

      - GRAYLOG_ROOT_PASSWORD_SHA2=564a5fcf9ceb22a225acd62ddf728fd92f666a02348fbabb2607454659c89965
      - GRAYLOG_HTTP_EXTERNAL_URI=http://88.198.189.243:9000/
    links:
      - mongodb:mongo
      - elasticsearch
    depends_on:
      - mongodb
      - elasticsearch
    ports:
      # Graylog web interface and REST API
      - 9000:9000
      # Syslog TCP
      - 1514:1514
      # Syslog UDP
      - 1514:1514/udp
      # GELF TCP
      - 12201:12201
      # GELF UDP
      - 12201:12201/udp
    networks:
      vpcbr:
        ipv4_address: 172.20.0.12

Graylog container log:

    2020-09-22 09:15:09,337 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical. Not configured to run on this node.
2020-09-22 09:15:09,337 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, running forever.
2020-09-22 09:15:09,345 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s].
2020-09-22 09:15:09,348 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s].
2020-09-22 09:15:09,348 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling every [10s].
2020-09-22 09:15:09,349 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.NodePingThread] periodical in [0s], polling every [1s].
2020-09-22 09:15:09,349 INFO : org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration - Legacy default stream has no connections, no migration needed.
2020-09-22 09:15:09,354 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.VersionCheckThread] periodical in [300s], polling every [1800s].
2020-09-22 09:15:09,354 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ThrottleStateUpdaterThread] periodical in [1s], polling every [1s].
2020-09-22 09:15:09,356 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:4, serverValue:65}] to mongo:27017
2020-09-22 09:15:09,362 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventPeriodical] periodical in [0s], polling every [1s].
2020-09-22 09:15:09,364 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.events.ClusterEventCleanupPeriodical] periodical in [0s], polling every [86400s].
2020-09-22 09:15:09,366 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ClusterIdGeneratorPeriodical] periodical, running forever.
2020-09-22 09:15:09,367 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:5, serverValue:66}] to mongo:27017
2020-09-22 09:15:09,368 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesMigrationPeriodical] periodical, running forever.
2020-09-22 09:15:09,374 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexRangesCleanupPeriodical] periodical in [15s], polling every [3600s].
2020-09-22 09:15:09,379 INFO : org.graylog2.shared.initializers.PeriodicalsService - Not starting [org.graylog2.periodical.UserPermissionMigrationPeriodical] periodical. Not configured to run on this node.
2020-09-22 09:15:09,379 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.ConfigurationManagementPeriodical] periodical, running forever.
2020-09-22 09:15:09,384 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:6, serverValue:67}] to mongo:27017
2020-09-22 09:15:09,385 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.LdapGroupMappingMigration] periodical, running forever.
2020-09-22 09:15:09,386 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.IndexFailuresPeriodical] periodical, running forever.
2020-09-22 09:15:09,402 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.periodical.TrafficCounterCalculator] periodical in [0s], polling every [1s].
2020-09-22 09:15:09,404 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog2.indexer.fieldtypes.IndexFieldTypePollerPeriodical] periodical in [0s], polling every [3600s].
2020-09-22 09:15:09,406 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.scheduler.periodicals.ScheduleTriggerCleanUp] periodical in [120s], polling every [86400s].
2020-09-22 09:15:09,412 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredSidecarsThread] periodical in [0s], polling every [600s].
2020-09-22 09:15:09,425 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.sidecar.periodical.PurgeExpiredConfigurationUploads] periodical in [0s], polling every [600s].
2020-09-22 09:15:09,438 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.views.search.db.SearchesCleanUpJob] periodical in [3600s], polling every [28800s].
2020-09-22 09:15:09,439 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.events.periodicals.EventNotificationStatusCleanUp] periodical in [120s], polling every [86400s].
2020-09-22 09:15:09,444 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread] periodical in [0s], polling every [3600s].
2020-09-22 09:15:10,037 INFO : org.graylog2.shared.initializers.JerseyService - Enabling CORS for HTTP endpoint

Elastic container log:

    [2020-09-22T09:19:03,734][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [aggs-matrix-stats]
[2020-09-22T09:19:03,735][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [analysis-common]
[2020-09-22T09:19:03,735][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [ingest-common]
[2020-09-22T09:19:03,735][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [ingest-geoip]
[2020-09-22T09:19:03,735][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [ingest-user-agent]
[2020-09-22T09:19:03,735][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [lang-expression]
[2020-09-22T09:19:03,735][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [lang-mustache]
[2020-09-22T09:19:03,735][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [lang-painless]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [mapper-extras]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [parent-join]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [percolator]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [rank-eval]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [reindex]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [repository-url]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [transport-netty4]
[2020-09-22T09:19:03,736][INFO ][o.e.p.PluginsService     ] [IydRrw7] loaded module [tribe]
[2020-09-22T09:19:03,737][INFO ][o.e.p.PluginsService     ] [IydRrw7] no plugins loaded
[2020-09-22T09:19:07,264][INFO ][o.e.d.DiscoveryModule    ] [IydRrw7] using discovery type [zen] and host providers [settings]
[2020-09-22T09:19:07,738][INFO ][o.e.n.Node               ] [IydRrw7] initialized
[2020-09-22T09:19:07,739][INFO ][o.e.n.Node               ] [IydRrw7] starting ...
[2020-09-22T09:19:07,872][INFO ][o.e.t.TransportService   ] [IydRrw7] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2020-09-22T09:19:07,893][WARN ][o.e.b.BootstrapChecks    ] [IydRrw7] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2020-09-22T09:19:10,970][INFO ][o.e.c.s.MasterService    ] [IydRrw7] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {IydRrw7}{IydRrw7lSBCRibR_7MIV_w}{RNANG_WEQ5CViJwdMfW7Aw}{localhost}{127.0.0.1:9300}
[2020-09-22T09:19:10,991][INFO ][o.e.c.s.ClusterApplierService] [IydRrw7] new_master {IydRrw7}{IydRrw7lSBCRibR_7MIV_w}{RNANG_WEQ5CViJwdMfW7Aw}{localhost}{127.0.0.1:9300}, reason: apply cluster state (from master [master {IydRrw7}{IydRrw7lSBCRibR_7MIV_w}{RNANG_WEQ5CViJwdMfW7Aw}{localhost}{127.0.0.1:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2020-09-22T09:19:11,027][INFO ][o.e.h.n.Netty4HttpServerTransport] [IydRrw7] publish_address {172.20.0.11:9200}, bound_addresses {0.0.0.0:9200}
[2020-09-22T09:19:11,028][INFO ][o.e.n.Node               ] [IydRrw7] started
[2020-09-22T09:19:11,189][WARN ][o.e.d.c.j.Joda           ] [IydRrw7] 'y' year should be replaced with 'u'. Use 'y' for year-of-era. Prefix your date format with '8' to use the new specifier.
[2020-09-22T09:19:11,476][INFO ][o.e.g.GatewayService     ] [IydRrw7] recovered [3] indices into cluster_state
[2020-09-22T09:19:12,164][INFO ][o.e.c.r.a.AllocationService] [IydRrw7] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[graylog_0][0]] ...]).
[2020-09-22T09:19:14,891][WARN ][o.e.d.r.a.a.i.RestGetMappingAction] [IydRrw7] [types removal] The parameter include_type_name should be explicitly specified in get mapping requests to prepare for 7.0. In 7.0 include_type_name will default to 'false', which means responses will omit the type name in mapping definitions.

 Mongo log

    2020-09-22T09:20:45.878+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-09-22T09:20:45.878+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=7513M,cache_overflow=(file_max=0M),session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2020-09-22T09:20:47.775+0000 I STORAGE  [initandlisten] WiredTiger message [1600766447:775861][1:0x7f414cb47a40], txn-recover: Main recovery loop: starting at 3/3119872
2020-09-22T09:20:47.988+0000 I STORAGE  [initandlisten] WiredTiger message [1600766447:988130][1:0x7f414cb47a40], txn-recover: Recovering log 3 through 4
2020-09-22T09:20:48.106+0000 I STORAGE  [initandlisten] WiredTiger message [1600766448:106856][1:0x7f414cb47a40], txn-recover: Recovering log 4 through 4
2020-09-22T09:20:48.197+0000 I STORAGE  [initandlisten] WiredTiger message [1600766448:197045][1:0x7f414cb47a40], txn-recover: Set global recovery timestamp: 0
2020-09-22T09:20:48.246+0000 I CONTROL  [initandlisten]
2020-09-22T09:20:48.246+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-09-22T09:20:48.246+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2020-09-22T09:20:48.246+0000 I CONTROL  [initandlisten]
2020-09-22T09:20:48.258+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2020-09-22T09:20:48.259+0000 I NETWORK  [initandlisten] listening via socket bound to 0.0.0.0
2020-09-22T09:20:48.260+0000 I NETWORK  [initandlisten] listening via socket bound to /tmp/mongodb-27017.sock
2020-09-22T09:20:48.260+0000 I NETWORK  [initandlisten] waiting for connections on port 27017
2020-09-22T09:20:48.341+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42338 #1 (1 connection now open)
2020-09-22T09:20:48.341+0000 I NETWORK  [conn1] received client metadata from 172.20.0.12:42338 conn1: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }
2020-09-22T09:20:48.347+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42340 #2 (2 connections now open)
2020-09-22T09:20:48.348+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42342 #3 (3 connections now open)
2020-09-22T09:20:48.348+0000 I NETWORK  [conn2] received client metadata from 172.20.0.12:42340 conn2: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }
2020-09-22T09:20:48.348+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42344 #4 (4 connections now open)
2020-09-22T09:20:48.348+0000 I NETWORK  [conn3] received client metadata from 172.20.0.12:42342 conn3: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }
2020-09-22T09:20:48.348+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42346 #5 (5 connections now open)
2020-09-22T09:20:48.348+0000 I NETWORK  [conn4] received client metadata from 172.20.0.12:42344 conn4: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }
2020-09-22T09:20:48.348+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42348 #6 (6 connections now open)
2020-09-22T09:20:48.348+0000 I NETWORK  [conn5] received client metadata from 172.20.0.12:42346 conn5: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }
2020-09-22T09:20:48.349+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42350 #7 (7 connections now open)
2020-09-22T09:20:48.349+0000 I NETWORK  [conn6] received client metadata from 172.20.0.12:42348 conn6: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }
2020-09-22T09:20:48.349+0000 I NETWORK  [listener] connection accepted from 172.20.0.12:42352 #8 (8 connections now open)
2020-09-22T09:20:48.349+0000 I NETWORK  [conn7] received client metadata from 172.20.0.12:42350 conn7: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }
2020-09-22T09:20:48.349+0000 I NETWORK  [conn8] received client metadata from 172.20.0.12:42352 conn8: { driver: { name: "mongo-java-driver|legacy", version: "3.12.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.9.0-12-amd64" }, platform: "Java/Oracle Corporation/1.8.0_265-b01" }

Graylog proxy conf:

    server {
  listen 80;
  server_name graylog.com;
  location / {
    return 301 https://$host$request_uri;
  }

  location ^~ /.well-known/acme-challenge {
    alias /var/lib/dehydrated/acme-challenges;
   # allow all;
   # default_type "text/plain";
  }
}
server {
  listen 443 ssl;
  ssl_certificate /var/lib/dehydrated/certs/graylog.whaleapp.team/fullchain.pem;
  ssl_certificate_key /var/lib/dehydrated/certs/graylog.whaleapp.team/privkey.pem;

  server_name graylog.com;

  location / {
    proxy_read_timeout 300;
    proxy_set_header        Host $host:$server_port;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        X-Graylog-Server-URL http://$server_name/api;
    proxy_pass          http://172.20.0.12:9000;
    #proxy_read_timeout  90;
    #proxy_redirect off;
    proxy_redirect      http://172.20.0.12:9000 https://graylog.com;
    proxy_http_version 1.1;
    proxy_request_buffering off;
  }
}
offlinejke commented 3 years ago

Fixed with proxy_set_header X-Graylog-Server-URL https://$server_name;