dwbutler / logstash-logger

Ruby logger that writes logstash events
MIT License
456 stars 119 forks source link

Fails to send logs: special settings to add to logstash conf file ? #147

Open belgoros opened 6 years ago

belgoros commented 6 years ago

Do I need to add special settings to put into the Logstash configuration file or any others to make it work ? Can firewall or proxy settings block that ? Actually, I can see logs written only pic.log file.

I have the following gems declared in Rails API Gemfile:

ruby '2.5.0'
gem 'rails', '~> 5.2.0'
...

group :development, :pic, :staging do
  gem 'lograge',         '~> 0.10.0'
  gem 'logstash-event',  '~> 1.2'
  gem 'logstash-logger', '~> 0.26.1'
...
end

Here is the settings in config/environement/pic.rb:

...
config.lograge.enabled = true
  config.lograge.base_controller_class = 'ActionController::API'
  config.lograge.formatter = Lograge::Formatters::Logstash.new
  config.lograge.logger = LogStashLogger.new(
    type: :multi_logger,
    outputs: [
      { type: :file, path: "log/#{Rails.env}.log", formatter: ::Logger::Formatter },
      { type: :tcp, host: ENV['logstash_host'], port: ENV['logstash_port'], formatter: :json }
      ],
    verify_hostname: false
  )

  config.lograge.custom_options = lambda do |event|
    { name:'decastore-pic', host: event.payload[:host] }
  end

there are no logs when I tried to see it in Kibana dashboard (I hoped to filter by decastore by setting name option):

screen shot 2018-08-16 at 11 15 47

Even when trying to run the following ruby script:

require 'logstash-logger'

logger = LogStashLogger.new(
  type: :tcp,
  host: 'logstash_host_name',
  port: 3332,
  verify_hostname: false
)

logger.debug message: 'decastore-draft', foo: 'bar'
logger.info 'decastore-draft => test'

nothing is sent either.

dwbutler commented 6 years ago

Have you read this note: https://github.com/dwbutler/logstash-logger#logstash-never-receives-any-logs and tried setting sync: true?

belgoros commented 5 years ago

@dwbutler Thank you, I've tried to set sync: true in my staging.rb file:

config.log_formatter = ::Logger::Formatter.new
...
if ENV['RAILS_LOG_TO_STDOUT'].present?
    config.log_level = :info
    config.lograge.enabled = true
    config.lograge.base_controller_class = 'ActionController::API'
    config.lograge.formatter = Lograge::Formatters::Logstash.new

    config.lograge.custom_options = lambda do |event|
      exceptions = %w[controller action format id]
      {
        time: event.time,
        stack: 'decastore',
        host: event.payload[:host],
        params: event.payload[:params].except(*exceptions)
      }
    end

    config.lograge.logger = LogStashLogger.new(
      type: :tcp,
      host: ENV['logstash_host'],
      port: ENV['logstash_port'],
      verify_hostname: false,
      sync: true
    )
  end

I also tries to just run the following in rails console:

logger = LogStashLogger.new(type: :tcp, host: 'logstash-host.org', port: 3332, verify_hostname: false, sync: true)
logger.info 'decastore => test port 3332'

It fails both with udp and tcp type values.