search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.41k stars 1.07k forks source link

Problem with extracting variables #4158

Closed DodgeCamaro closed 7 years ago

DodgeCamaro commented 7 years ago

Expected Behavior

When a request comes, the mail should receive a notification with short information provided by the user.

Format message:

You have a new request
Date: 2017-09-14T18:54:23.671Z
Firstname: Jack
Lastname: Black
Phone: +12345678900
Site page: https://site.com/example/page
IP-adress: 000.00.00.00

I propose to create a separate tab with variables for their customization and call it "Varibles" with the items "Global" and "Local". "Global" for Inputs, and "Local" for the cluster Graylog.

Current Behavior

I have this alerts

##########
Alert Description: Stream received messages matching <user:"Black"> (Current grace time: 0 minutes)
Date: 2017-09-14T18:13:14.226Z
Stream ID: 59b92df92ab79c0001201034
Stream title: New Stream
Stream description: Maybe
Alert Condition Title: Condition_test
Stream URL: Please configure 'transport_email_web_interface_url' in your Graylog configuration file.

Triggered condition: a33229ec-cc9a-4577-84dd-0a58cc310de4:field_content_value={field: user, value: Black, grace: 0, repeat notifications: false}, stream:={59b92df92ab79c0001201034: "New Stream"}
    ##########

Last messages accounting for this alert:

Or this

##########
Alert Description: Dummy alert to test notifications
Date: 2017-09-14T18:18:34.009Z
Stream ID: 59b92df92ab79c0001201034
Stream title: New Stream
Stream description: Maybe
Alert Condition Title: Test Alert
Stream URL: Please configure 'transport_email_web_interface_url' in your Graylog configuration file.

Triggered condition: 5b28ae34-ef36-4600-860d-64b576301d65:dummy={Dummy alert to test notifications}, stream:={59b92df92ab79c0001201034: "New Stream"}
    ##########

 <No backlog>

I want take information an example that is written in "Expected Behavior"

You have a new request
Date: 2017-09-14T18:54:23.671Z
Firstname: Jack
Lastname: Black
Phone: +12345678900
Site page: https://site.com/example/page
IP-adress: 000.00.00.00

Steps to Reproduce (for bugs)

  1. Edit graylog.conf
password_secret = secretpass
root_password_sha2 = secretpasssha2
root_email = mail@mail.com
root_timezone = Europe/Kiev 
rest_listen_uri = https://0.0.0.0:9000/api/
web_listen_uri = https://0.0.0.0:9000/
elasticsearch_hosts = http://elasticsearch:9200
elasticsearch_compression_enabled = true
transport_email_enabled = true
transport_email_hostname = smtp.gmail.com
transport_email_port = 465 
transport_email_use_auth = true
transport_email_use_tls = true
transport_email_use_ssl = true
transport_email_auth_username = mail@mail.com
transport_email_auth_password = password
transport_email_subject_prefix = [graylog] 
transport_email_from_email = graylog@example.com
transport_email_web_interface_url = https://site.com

Other settings is default.
  1. Create nginx.conf
user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    proxy_hide_header X-Powered-By;
    add_header X-Frame-Options DENY;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";

    #Strict Transport Security
    add_header Strict-Transport-Security "max-age=63072000; preload" always;

    server_tokens off;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  off;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    gzip on;
    gzip_disable "msie6";
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;

    server {
        listen       80;
        listen       [::]:80;
        server_name  site.com;
        return 301 https://site.com$request_uri;
    }

    server {
        listen       443 ssl http2;
        listen       [::]:443 ssl http2;
        server_name  site.com;
        resolver 8.8.4.4 8.8.8.8 valid=300s;
        resolver_timeout 10s;

        ssl_stapling on;
        ssl_stapling_verify on;
        ssl_certificate /etc/ssl/private/fullchain.pem;
        ssl_certificate_key /etc/ssl/private/privkey.pem;
        ssl_trusted_certificate     /etc/ssl/private/chain.pem;
        ssl_dhparam /etc/pki/nginx/dhparam.pem;

        # Set up preferred protocols and ciphers. TLS1.2 is required for HTTP/2
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!aNULL:!ADH:!AECDH:!MD5;

        # This is a cache for SSL connections
        ssl_session_cache shared:SSL:10m;
        ssl_session_timeout 60m;

        access_log  off;

        if ( $http_user_agent ~* (nmap|nikto|wikto|sf|sqlmap|bsqlbf|w3af|acunetix|havij|appscan) ) {
            return 444;
        }

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Graylog-Server-URL https://$server_name/api;
            proxy_pass       http://graylog:9000;
         }

        location /log {
            proxy_set_header Host $http_host;
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Graylog-Server-URL https://$server_name/gelf;
            proxy_pass       http://graylog:12201/gelf;
         }
    }
}
  1. Create certificate in letsencrypt
  2. docker-compose up -d 
## docker-compose.yml
version: '2'
services:
  proxy:
    image: nginx
    container_name: proxy
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./proxy/nginx.conf:/etc/nginx/nginx.conf
      - /etc/letsencrypt/live/site.com/privkey.pem:/etc/ssl/private/privkey.pem
      - /etc/letsencrypt/live/site.com/fullchain.pem:/etc/ssl/private/fullchain.pem
      - /etc/letsencrypt/live/site.com/chain.pem:/etc/ssl/private/chain.pem
      - ./proxy/ssl/dhparam.pem:/etc/pki/nginx/dhparam.pem
    depends_on:
      - graylog
    mem_limit: 1g
    networks:
      - graylog

  mongodb:
    image: mongo:3
    container_name: mongo
    volumes:
      - /db:/data/db
    mem_limit: 2g
    restart: always
    networks:
      - graylog

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.5.1
    container_name: elasticsearch
    volumes:
      - es_data:/usr/share/elasticsearch/data
    environment:
      - http.host=0.0.0.0
      - transport.host=localhost
      - network.host=0.0.0.0
      # Disable X-Pack security: https://www.elastic.co/guide/en/elasticsearch/reference/5.5/security-settings.html#general-security-settings
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    mem_limit: 1g
    restart: always
    networks:
      - graylog

   graylog:
    image: graylog/graylog:latest
    container_name: graylog
    volumes:
      - ./graylog/config:/usr/share/graylog/data/config
      - graylog_journal:/usr/share/graylog/data/journal
    environment:
      - GRAYLOG_PASSWORD_SECRET=secretpass
      - GRAYLOG_ROOT_PASSWORD_SHA2=sha2secretpass
      - GRAYLOG_WEB_ENDPOINT_URI=https://site.com/api
    links:
      - mongodb:mongo
      - elasticsearch
    depends_on:
      - mongodb
      - elasticsearch
    mem_limit: 2g
    restart: always
    networks:
      - graylog

volumes:
  es_data:
    driver: local
  graylog_journal:
    driver: local

networks:
  graylog:
    driver: bridge

  1. Create Stream and add Stream Rules with this option Field source must contain https://

  2. Create Global Inputs GELF HTTP with default settings, on port 12201

  3. Create Conditions Configuration: Alert is triggered when messages matching <source: "https://"> are received. Grace period: 0 minutes. Including last message in alert notification. Configured to repeat notifications.

  4. Create Notifications

##########
Alert Description: ${check_result.resultDescription}
Date: ${check_result.triggeredAt}
Stream ID: ${stream.id}
Stream title: ${stream.title}
Stream description: ${stream.description}
Alert Condition Title: ${alertCondition.title}
${if stream_url}Stream URL: ${stream_url}${end}

Triggered condition: ${check_result.triggeredCondition}
##########

${if backlog}Last messages accounting for this alert:
${foreach backlog message.fields}${message.fields.status}

${end}${else}<No backlog>
${end}
  1. Send information in JSON from site to Graylog.
{
        "version":       "1.1",
        "full_message":  {"data": "message data"},
        "host":          "https://example.com",
        "short_message": "uid",
        "_email":        "j.doe@eample.com",
        "_step":         "step_1",
        "_status":       "success",
        "_agent":        "empty"
    }

Context

I want to get the function that is in the greylogue out of the box.

I spent a lot of time looking for a solution to this problem, but I did not find it.

Your Environment

joschi commented 7 years ago

@DodgeCamaro You can access arbitrary message fields in the template used in the Email Alarm Callback as described in http://docs.graylog.org/en/2.3/pages/streams/alerts.html#email-alert-notification

We are using GitHub issues for tracking bugs in Graylog itself, but this doesn't look like one. Please post this issue to our discussion forum or join the #graylog channel on freenode IRC.

Thank you!

DodgeCamaro commented 7 years ago

@joschi Please answer on my question here or on the forum website. My question on the forum remains unanswered for three days already !!!

joschi commented 7 years ago

@DodgeCamaro There are also professional support services if you require specific SLA: https://www.graylog.org/enterprise