LDAP search input plugin

[jsvd]

Migrated from: https://logstash.jira.com/browse/LOGSTASH-1772

Hello,
I had to create an LDAP search input plugin for my personal needs (in order to monitor changes of a group). I share it there in case it can help somebody.
You just have to add this line to logstash.gemspec:
  gem.add_runtime_dependency "jruby-ldap", "~>0.0.2"

# encoding: utf-8
require "logstash/inputs/base"
require "logstash/namespace"
require "socket"

# Perform an LDAP Search
#
# Example:
#
#     input {
#       LDAPSearch {
#         host => "myLDAPServer"
#         dn => "myDN"
#         password => "myPassword"
#         filter => "CN=HR Group"
#         base => "OU=Europe,o=Acme"
#         interval => 10
#       }
#     }
class LogStash::Inputs::LDAPSearch < LogStash::Inputs::Base

  config_name "LDAPSearch"
  milestone 1

  # LDAP parameters
  config :host, :validate => :string, :required => true
  config :dn, :validate => :string, :required => true
  config :password, :validate => :password, :required => true
  config :filter, :validate => :string, :required => true
  config :base, :validate => :string, :required => true
  config :port, :validate => :number, :default => 389
  # Polling interval
  config :interval, :validate => :number, :default => 10  

  public
  def register
    require "base64"
    require "ldap"
  end # def rbgister

  public
  def run(queue)

    @host = Socket.gethostbyname(@host).first
    conn = LDAP::Conn.new(@host, @port)
    conn.bind(@dn, @password.value)

    begin
      @logger.debug("Executing LDAP search base='#{@base}' filter='#{@filter}'")
      loop do
        conn.search(@base, LDAP::LDAP_SCOPE_SUBTREE, @filter) { |entry|
          event = LogStash::Event.new
          event["host"] = @host
          decorate(event)
          entry.get_attributes.each do |attr|
            values = entry.get_values(attr)
            values = values.map { |value|
              (/[^[:print:]]/ =~ value).nil? ? value : Base64.strict_encode64(value)
            }
            event[attr] = values
          end
          queue << event
        }
        sleep @interval
      end # loop
    rescue LDAP::ResultError => ex
      logger.error("LDAP search error: #{ex}\n#{ex.backtrace}")
      sleep @interval
      retry
    end # begin/rescue
  end # def run
end # class LogStash::Inputs::LDAPSearch

[electrical]

We could create a plugin for this indeed.

[kofemann]

+1

[missnebun]

+1

[ptitloup]

This is currently under development : https://github.com/SemmLille/logstash-input-LDAPSearch

[jsvd]

is it based on the same code base or are the same people involved? if so I could close this

[ptitloup]

I started with this code but i had to modify it to return specific attributs. I just start today and I have some trouble with bundle install.

:/opt/logstash-1.5.0/logstash-input-LDAPSearch# bundle install Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine. There was a NoMethodError while loading logstash-input-LDAPSearch.gemspec: undefined method `metadata=' for #Gem::Specification:0x7fb7e5253740 from /opt/logstash-1.5.0/logstash-input-LDAPSearch/logstash-input-LDAPSearch.gemspec:20

If you have some idea...

Thanks

[ptitloup]

I found a solution to solve my problem and run bundle install. metadata appear only with ruby 2.. So to install it on debian, i used rvm.

#gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

#curl -sSL https://get.rvm.io | bash -s stable --ruby

and after, to use it :

source /usr/local/rvm/scripts/rvm

ruby -v

ruby 2.2.1p85 (2015-02-26 revision 49769) [x86_64-linux]

[untergeek]

@ptitloup I see that you have published this plugin to rubygems. Did you want it to be a part of the logstash-plugins repository? No worries if you don't. If you do, you can even remain the project maintainer if you want.

Please let us know.

[ptitloup]

Hello, Yes, why not, with a great pleasure. Thanks for proposal Ptitloup

Le 11/03/2016 18:27, Aaron Mildenstein a écrit :

@ptitloup https://github.com/ptitloup I see that you have published this plugin to rubygems. Did you want it to be a part of the logstash-plugins https://github.com/logstash-plugins repository? No worries if you don't. If you do, you can even remain the project maintainer https://www.elastic.co/guide/en/logstash/2.2/community-maintainer.html if you want.

Please let us know.

— Reply to this email directly or view it on GitHub https://github.com/elastic/logstash/issues/2523#issuecomment-195466201.

Signature Nicolas CAN

Nicolas CAN Ingénieur développement et intégration d'application Responsable informatique documentaire Service WEB-ENT - Centre de Ressources Informatiques Tel : (+33) 03 20 33 60 73 Courriel : nicolas.can@univ-lille1.fr mailto:nicolas.can@univ-lille1.fr

université lille1 sciences et technologie http://www.univ-lille1.fr

Université de Lille, Sciences et Technologies Service Commun de Documentation - Bat. A5 Boulevard Paul Langevin 59655 Villeneuve d'Ascq Cedex

http://www.univ-lille1.fr

[sok-pomaranczowy]

Has this project became part of logstash-plugins? I could not find it there. Is there by any chance some work that could be done?

[andrewvc]

@sok-pomaranczowy if you'd like to adopt / maintain the plugin that'd be awesome. @acchen97 would be the one to talk to about that.

[sok-pomaranczowy]

@andrewvc thanks. perhaps I'd be interested but I need to look at the code first and read your guide to contributing. This is the repo we would be talking about? https://github.com/SemmLille/logstash-input-LDAPSearch

[acchen97]

@sok-pomaranczowy that link is correct, but the plugin is quite old at this point and would require quite a bit of change. What use case were you looking to conduct here? Is it to monitor the LDAP change log?

[sok-pomaranczowy]

Im sorry for the late reply. I wanted to use it to implement search on my LDAP instance by doing LDAP->Logstash->Elasticsearch. Seeing that it is not maintained I thought it would be a great chance to get into OSS with both something small and useful.

[JPvRiel]

Just to add a similar but distinct use case for an LDAP enrichment filter into this idea because there's commonality and room for code sharing. There are two cases:

• input plugin: Poll LDAP a log query results as an input (e.g. monitor for group changes, as per original author's intent)
• filter plugin: query LDAP based on the content of an existing field (e.g. SamAccountName, SID ,etc) and add, replace, etc another metadata field (e.g. UserPrincipalName, DistinguishedName, etc)

I think the 2nd use case can also be very valuable given logs are often filled with unfriendly identifiers (e.g. Active Directory/Windows has an SID) meanwhile the search use case might want to target a real username...

Both the input plugin and filter plugin could borrow from or leverage the same underlying ruby ldap query component that should hopefully include a caching mechanism to improve performance.

[JPvRiel]

By the way, this looks promising: https://github.com/Transrian/logstash-filter-ldap (haven't tested it out yet). Important features, like an in-memory cache suggest the authors did it properly to handle fairly high load and improve performance.