No support for config reload with stdin input plugin

[rocketraman]

I am using config reloading for testing my configuration. When I start up I get the message:

[2017-02-02T11:33:09,198][ERROR][logstash.agent           ] Logstash is not able to start since configuration auto reloading was enabled but the configuration contains plugins that don't support it. Quitting... {:pipeline_id=>"main", :plugins=>[LogStash::Inputs::Stdin]}

I am using logstash 5.2.0.

[John1Q84]

I have same problem with my test logstash box in my logstash.yml file, plugin part is commented, it should be adopted as the defult setting,,

[7DataBits]

Autoreload is disabled on purpose for the stdin input plugin.

# When a configuration is using this plugin
# We are defining a blocking pipeline which cannot be reloaded
def self.reloadable?
  false
end

[rocketraman]

Autoreload is disabled on purpose for the stdin input plugin.

Well, yes it is, but that sucks: I'd most use stdin for testing changes to my config. And reload is most useful for the same purpose -- with reload, I can make config changes and easily iterate on the result.

Perhaps a workaround can be documented, like using a file input plugin pointing to a named pipe (I haven't tried this, but I don't see why it wouldn't work)?

[Kamarey]

Same here. A possible solution is to create dedicated plugin for testing purpose with reload support

[jsvd]

Unfortunately that's just how java works, the stdin channel can't be closed and reopened. We already have extra code that allows logstash to interrupt the read of the stdin's underlying channel so that logstash can terminate gracefully.

The way the interrupt happens is by calling close and catching the exception, but once that is done, it's not possible (from what we know) to reopen stdin until the JVM is restarted.

Patches, suggestions or new ideas are more than welcome. The requirement is simple: it must be possible for a thread to read stdin, be told to stop reading, and be able to read again (this last one is what we don't support).

[rocketraman]

I'm not sure what rereading standard in has to do with reconfiguration of logstash with respect to things that are done downstream. Is the entire pipeline configured as one big stream and reconfiguration just reloads the entire thing by recreating everything from scratch? Or can the standard input plugin just change its reference to the next entry in the pipeline?

[jsvd]

Correct, as of now pipelines are reloaded as a whole, we don't support partial reloading of the filter or output sections only. We aim to have that in the future but it is outside the scope of this plugin.

At the moment the only work around is having a separate pipeline sending from stdin to..lumberjack or TCP for example, and reloading a second pipeline that uses beats input or TCP input with the filters you want to test.

[jsvd]

Also, just for reference, there is an issue open for adding a simulate API to logstash https://github.com/elastic/logstash/issues/7832 , that you allow you to test events against the filter section of a pipeline configuration