Simplified Logstash offline plugin install

[andrewvc]

While we currently support offline plugin installation, it has had issues with consistently working. It is also tough to work with and test, which makes development hard. In short, it's a pain for both users and developers.

I propose that we take a new approach for offline plugin installation. We simply change our officially recommended method of installation to either:

1. Download logstash.tar.gz to an internet connected box
2. Install/upgrade all required additional plugins
3. Tar up the directory
4. Use this tarball on all airgapped machines

or:

1. Download logstash.tar.gz to an internet connected box
2. Install/upgrade all required additional plugins
3. run a command to generate a tar/deb/rpm/zip or whatever locally
4. Distribute your own Logstash to airgapped instances

Thoughts?

[praseodym]

I’ve never have problems with logstash-plugin prepare-offline-pack, which I mainly use to create reproducible, fast Docker image builds. For plugin development I’ve created a small Rake task to pack the built plugin gem into an offline pack zip, which can be easily installed in a Docker container for testing.

Reverting to preparing tarballs would be a real hassle, especially if one needs to deploy Logstash instances with different sets of plugins installed.

[guyboertje]

I'm not sure about users ops workflow here, specifically, the kind of access they have to the offline machine.

Another alternative maybe (*nix example)...

1. Stop offline LS instance.
2. rsync -avz -e ssh --exclude 'data' remoteuser@remotehost:/path/to/logstash /local/path/to/logstash
3. Start offline LS instance.
4. Install or upgrade plugins.
5. Stop offline LS instance.
6. rsync -avz -e ssh /local/path/to/logstash remoteuser@remotehost:/path/to/logstash
7. Start offline LS instance.

Because we don't want to rsync copy logs, PQ and DLQ files we could provide a file with the rsync exclude folders to use as --exclude-from 'logstash-rsync-excludes.txt'