Fluentd install comments and potential improvements

[karolh2000]

This pull request is meant to improve the log analytics installation experience. The changes in the PR should be treated as suggestions as I’m not very familiar with the log-analytics functionality. The PR contains the following changes:

1. Interactive Fluentd installer with the following functionality:

   • Downloads the github repo and all dependencies [optional]
   • Checks if the Fluentd requirements are met and updates the OS if needed [optional]
     • Installs/Updates Fluentd as a service depending on Linux distro (Centos and Amazon is supported, more to come).
     • Updates the log files/folders permissions [optional].
     • Installs Fluentd plugins (Splunk, Datadog, Elastic, etc)[optional].
     • Starts and enables the td service [optional].
     • Suggests next step (link to the fluentd configuration steps based on the installed plugin).

2. Changes in the log-analytics README.md.

3. General comments not directly related/included to the PR changes.

   • [Splunk repo] - https://github.com/jfrog/log-analytics-splunk README.md

     • “Configure HEC to accept data” - this steps should mention to write down the HEC port used later the steps (default is 8088)

       • “Configure new HEC token” - this doesn't sound right?:

         5 Enter a "Name" in the textbox

       • “Splunk App” - JFrog app can be installed directly from the Splunk store instead of a file (app) that needs to be uploaded/installed later so some steps could be removed.

   • [Elastic] - https://github.com/jfrog/log-analytics-elastic

     • The fluentd steps in README.md seem to be unnecessary since the same info is available at the log-analytics repo (probably hard to maintain the some content in multiple locations).

   • [Prometheus] - https://github.com/jfrog/log-analytics-prometheus

     • IMHO this one is the most challenging
     • It’d be great if we provide more information about the k8s/yaml files and their purpose.
     • I tried to set up the environment using minikube/helm but eventually gave up (some of the yaml files were throwing errors (e.g Unexpected args:... + LB service is not supported in minikube).

   • General comments that apply to all README.md/steps

     • I'd be great if we could provide info on how to troubleshoot in case of problems, e.g where to find relevant logs, etc.
     • In every section related to fluentd.conf file we should remind that JF_PRODUCT_DATA_INTERNAL needs to be specified, otherwise fluentd won't work.
     • It’d be worth mentioning it where the fluentd conf file is located (e.g for td-service service, etc).

Please let me know if you have any questions.

[github-actions[bot]]

CLA Assistant Lite bot All contributors have signed the CLA ✍️

[karolh2000]

I have read the CLA Document and I hereby sign the CLA

[karolh2000]

@peters95 John, @MahithaB Mahitha. Please take a look and my findings/suggestions/changes. If you are OK with the changes and the script then great if not or additional work is needed please let me know. These changes are merely suggestions for what would be in in my opinion useful to make the process smoother. Thanks