vanthome / winston-elasticsearch

An elasticsearch transport for winston
MIT License
270 stars 132 forks source link

Winston and Elasticsearch in different docker container #106

Closed ducho closed 4 years ago

ducho commented 4 years ago

Hi! First, I need to log all HTTP requests/responses, but this is problem with gelf/logstash because "\n" and my res/req body is exploded to several rows and its problem to read data in Kibana. I decided to try winston but I don't know if I need a morgan too.

I have a problem with logging messages to different container maybe because node IP address specified in ClientOps. Currently I'm using ELK stack in my app with gelf driver for logstash. All messages to STDERR and STDOUT (console.log) are logged throught logstash to ES.

var winston = require('winston');
var Elasticsearch = require('winston-elasticsearch');

var esTransportOpts = {
  level: 'error',
  clientOpts: {
    node: 'http://172.23.0.2:9200'
  }
};

var logger = winston.createLogger({
  transports: [
    new Elasticsearch(esTransportOpts)
  ]
});

logger.info('duchov test');

My docker configuration:

services:
  api:
    build:
      context: ./api
      args:
        - NODE_ENV=development
    command: node-dev --inspect=0.0.0.0:9229 --poll server.js
    ports:
      - "3000:3000"
      - "9229:9229"
    volumes:
      - ./api:/opt/app
      - notused:/opt/app/node_modules
    environment:
      - NODE_ENV=development
    depends_on:
      - mysql
      - influxdb
      - fakesqs
    logging:
      driver: gelf
      options:
        gelf-address: udp://172.23.0.3:12201
        tag: api

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.1.1
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
    networks:
      - logging-network

  logstash:
    image: docker.elastic.co/logstash/logstash:7.1.1
    depends_on:
      - elasticsearch
    ports:
      - 12201:12201/udp
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf:ro
    networks:
      - logging-network

  kibana:
    image: docker.elastic.co/kibana/kibana:7.1.1
    depends_on:
      - logstash
    ports:
      - 5601:5601
    networks:
      - logging-network
volumes:
    notused:

networks:
  logging-network:
    driver: bridge

Is it possible to combine winston-es with standard ELK configuration. Thanks for your possible advice! Regards. Dusan.

vanthome commented 4 years ago

This module is about sending logs directly to ES and not through Logstash so you are asking in the wrong place. The only thing I can tell that looks suspicious is that you provide this node: node: 'http://172.23.0.2:9200' and about your ES instance I'm not sure whether that IP and Port are exposed. Usually you don't specify the host as IP address in docker but with a host name.