search

logstash-plugins / logstash-filter-csv

Apache License 2.0
15 stars 41 forks source link

limit workers to 1 when autodetect_column_names is enabled #72

Open robin13 opened 5 years ago

robin13 commented 5 years ago

As fixing https://github.com/logstash-plugins/logstash-filter-csv/issues/65 is a big issue requiring fundamental work on logstash to synchronize between threads, would it be possible for the interim to

  1. Document that when autodetect_column_names is true pipeline.workers must be set to 1
  2. Have a startup check such that if autodetect_column_names is true and pipeline.workers is >1 then startup will fail (similar to the way the aggregate filter does).
colinsurprenant commented 4 years ago

the new csv codec should be more appropriate for this, when paired with the file input a new codec instance will be used per file.