Enable to rename hardcoded index names in visualizations for #69

[radoondas]

This modification expect a configuration file (.beatconfig), where necessary renaming is defined. Configuration file can be empty or doesn't need to exist. Script doesn't touch original files, it creates temp files instead. It works with other custom visualizations - just add your names and your set to go. It has also 'fallback' option if there is no configuration file - it is questionable what to do - do we 'sed' when there is no change? Also 'tokens' can be configurable? This solution requires further discussion.

[andrewkroh]

I think we should have feature parity between load.sh and load.ps1 so whatever we decide to do here should be implemented in the load.ps1 as well.

[radoondas]

I agree. Unfortunatelly, I am not able to do it for windows script yet.

[ruflin]

One idea here could be that we separate pushing the dashboards to elasticsearch from generating / updating them. I like that load.sh (and load.ps1) are scripts that can be executed on almost any machine without any requirements. The dynamic generation of updated template is probably much less common and is done by people that understand in detail what is going on. Do you guys think it would be an issue to require for these people having python installed for the first step generation?

[damm]

Python is available pretty much everywhere; you could make a windows release with a windows environment preinstalled in the windows zip for ease of use.

No need to limit yourself

[radoondas]

I have opposite feeling. To me, it is best to keep it as simple as possible and as native to OS as possible (UX or WIN). In production env, the number of available programming interfaces is limited and I can usually only dream of python or ruby. Even OS's perl is usually outdated. But shell is always available. Packing also python to the package will not work correctly due to different versions of OS.

I suggest to keep it shell based (bash or even better ksh) because of different *UX environment it can run on. I will also test this script on Solaris due to compatibility, later when I have some time left.

[monicasarbu]

@radoondas Thank you for your contribution. LGTM. I would say it's ok for now to read from .beatconfig only for the unix systems. We can implement it ourselves for Windows or maybe someone else from the community will be faster. Can you please rebase, so we can merge your PR?

[radoondas]

@monicasarbu I have fixed my PR. It is now with sync. Only a few questions remains for me. 1, Where to document how to use it. 2, What do we do if there is no need to 'sed' for changes. It is not necessary to 'sed' if there is no change against original. 3, Do we change 'tokens' ? It is simpler if the are as original, then there is no extra effort while regenerating fresh searches or beat.

[ruflin]

I'm also ok moving forward here. I think we need to rethink this in the long term as templates will get more complex. I'm thinking about loading templates through beats directly and the values above directly in the beats config file (just and idea so far).

About the next steps:

• Docs: I suggest to add it to the README.md for the moment and we transfer it to the elastic docs in a second stage (@dede ?)
• Sed: Not 100% sure I get this point. Is there a side affect if sed is used but has not affect? If not, it shouldn't matter
• +1 on keeping the originals as default. Otherwise we have to come up with some values and they will be wrong in most of the cases. It is good to provide an example, for example for Logstash.

[radoondas]

I'm fine with the next steps.

I should explain more this 'sed' thing I mentioned earlier. What I mean is that if there is no change in the names as they are hardcoded in templates, then we will as a side effect create temp file without changes against original file, this file import and then temp file delete. This is the only side effect which have no real impact.

[monicasarbu]

@radoondas We recently decided to move all the dashboards related to a Beat to the beats repository and starting with the next release 5.0.0-alpha2, we will not use the beats-dashboards anymore.

Starting with 5.0.0-alpha2, the load.sh script will be replaced with the import_dashboards.sh from https://github.com/elastic/beats/tree/master/dev-tools and the script together with the Kibana dashboards will be part of the Beat package, so you will be able to import the Kibana dashboards only for a single Beat.

As the script for importing the dashboards is getting more and more complex, we are planning to write a version in Golang instead of expanding the bash and powershell scripts. The biggest advantage is that we need to maintain only one script, not two scripts.

Thank you for contributing. The change you did is already used by other users. Can you please rebase, so we can merge your PR?

[radoondas]

@monicasarbu I understand and agree on approach with just one script :) And OK, I'll rebase as soon as I can.

[radoondas]

@monicasarbu I had to create another PR #117 due to complexity of the rebase. It took me much less time to do another PR from up-to-date-branch. Sorry for any inconvenience.

[monicasarbu]

@radoondas no problem, I merged the other PR.