logstash-plugins / logstash-input-jdbc

Logstash Plugin for JDBC Inputs
Apache License 2.0
449 stars 187 forks source link

Provide way for user to supply one or more Sequel Extensions #301

Open yaauie opened 6 years ago

yaauie commented 6 years ago

Since there exists a wide variety of SQL dialects, it would be useful if we allowed user to provide one or more Sequel "extensions" for their database connection. Sequel Extensions are capable of providing custom behaviour on top of a Sequel::Dataset or Sequel::Database, and are already in use internally with the Named Timezones and Pagination extensions. Extensions can modify how a Sequel::Dataset generates SQL, how statements are executed, or can modify other behaviour.

Allowing users to provide an arbitrary extension could be a good middle-ground to support database vendor-specific behaviour, without owning it in this repository (since we aren't experts in the intricacies of every SQL dialect).


One such example is Informix's syntax for limits and offsets, which are part of the SELECT clause:

SELECT [FIRST X] [SKIP Y]

An MIT-licensed adapter to support this syntax was previously a part of Sequel core, and could easily be ported to be an Extension (e.g., a module that can be included into an instance of Sequel::Dataset instead of a class that extends Sequel::Dataset). It was removed along with a host of other adapters because the maintainer of Sequel is not an expert in these vendors implementations and eschewed the maintenance load, while also steering the project toward independent extensions that modify behaviour.

As I envision it, allowing the user to provide a map of adapter names to paths to existing files. Something like:

input {
  jdbc {
    # ...
    sequel_extensions => {
      "informix_limit_support"     => "/path/to/informix_limit_support.rb"
      "informix_cursor_pagination" => "/path/to/informix_cursor_pagination.rb"
    }
  }
}

We could then iterate over this map, require the relevant files, and extend them onto our database or dataset. We would likely want to have safeguards (e.g., refusing to load the file if it is world-writable).