search

wazuh / wazuh-splunk

Wazuh - Splunk App
https://wazuh.com
GNU General Public License v2.0
50 stars 27 forks source link

Improve versions support #1308

Closed AlexRuiz7 closed 2 years ago

AlexRuiz7 commented 2 years ago

Describe the solution you'd like We need to study if it is possible to adjust our App to make it more flexible in terms of versions compatibility, by disregarding the minor versions of Splunk.

This would allow us to provide a single package per Splunk version (ie: 8.1, 8.2) instead of generating and supporting a package for each of the minor versions of Splunk (15 minors between 8.1 and 8.2).

Describe alternatives you've considered If this is not possible, then we can improve the current situation by automatizing the tags and packages generation, for example, using a script or CI/CD pipeline.

Additional context The main goal of this issue is to determine whether it is possible to install the same package along all the minor versions of a Splunk release.

Provide a report indicating it this is feasible or not, and if so, provide an estimation of costs in terms of time, workforce and side effect.

AlexRuiz7 commented 2 years ago

Versions mismatch warning toast message:

Conflict with the Wazuh app version. The version of the Wazuh app in your browser not correspond with the app version installed in Splunk. Please, clear your browser cache.

https://github.com/wazuh/wazuh-splunk/blob/d634cbc2ba1ba38f2a541dfc3ce1e737cb2a35e6/SplunkAppForWazuh/appserver/static/js/controllers/main/mainCtrl.js#L49-L55

AlexRuiz7 commented 2 years ago

Research

I've been reading the code and the logic implemented to check and compare the versions of Wazuh, and more importantly, the versions of Splunk, in order to determine if it is possible for us to generate and provide a single package for each minor version of Splunk, including all patch versions.

This is the sequence diagram representing the logic flow to compare the versions of the backend and the frontend components of the App.

Splunk_App_Versions

First, I quickly saw that our current logic for this is over-complicated and distributed along several configuration files and classes, which complicates the maintenance of the product, taking into account that all we need to do is read and compare two numbers, the revision numbers. Also, as the information about the App (version, revision, etc.) is distributed along several files, the pre-release flow is necessarily also complex and prone to errors.

The App information is currently distributed on the following files:

https://github.com/wazuh/wazuh-splunk/blob/d634cbc2ba1ba38f2a541dfc3ce1e737cb2a35e6/SplunkAppForWazuh/default/app.conf#L5-L15

https://github.com/wazuh/wazuh-splunk/blob/d634cbc2ba1ba38f2a541dfc3ce1e737cb2a35e6/SplunkAppForWazuh/default/package.conf#L1-L9

https://github.com/wazuh/wazuh-splunk/blob/d634cbc2ba1ba38f2a541dfc3ce1e737cb2a35e6/package.json#L26-L48

https://github.com/wazuh/wazuh-splunk/blob/d634cbc2ba1ba38f2a541dfc3ce1e737cb2a35e6/SplunkAppForWazuh/appserver/static/js/services/app-version/appVersionService.js#L1-L4

The app.conf is required by Splunk and contains essential information about the App, as the version and revision (build). I recommend continuing reading about the build property, as it has an impact on how Splunk caches static files (including JS code) for the Apps. Unfortunately, we aren't using this property as intended, as this should be our revision number.

On the other hand, the package.conf is not required at all by Splunk, which means that it has been generated by us as a convenience metadata file. The version and revision of the App is read from this file by the App's backend. The information contained in this file is already present in the package.conf file, so it is duplicated, and I greatly encourage updating the code to read from this file instead, which will allow us to remove the package.conf file, simplifying the pre-release flow as there is a configuration file less to maintain.

The main source of information should, to my criteria, be centralized on the package.json file. Here, we already have the target Wazuh version, Splunk version and App's revision number. On our pre-release flow, we use a script that propagates this information to the appVersionService.js file. We could improve this script to do the same for the app.conf file. Like this, all we had to do to generate a new package would be to increase the revision number by one and run the script to update both files.

Conclusion

We have talked about the target Wazuh version and the App revision, but not about the Splunk's version, which is what we are interested on. This is simply because this information is not read at all. Our version checking routine only checks that the Wazuh version installed on the forwarder matches the intended Wazuh version for the App, installed on the indexer node. The Splunk version is not taken into account in this comparison, and being objective, the Wazuh version shouldn't be either. Instead, the revision or build number, which is unique for each release, is what should be compared.

Required work

Estimated workforce:

AlexRuiz7 commented 2 years ago

In order to update the Python controllers to read the information from the app.conf file, we need to identify which interfaces are into play in terms of communication between the backend and the frontend.

The /manager/app_info returns the following JSON object, which we must preserve during the refactor:

{
    "revision": "4303",
    "version": "4.3.0",
    "splunk_version": "8.1.4"
}

On a second iteration, this could be improved to:

{
    "revision": "4303",
    "version": "4.3.0",
    "splunk": "8.1.x | 8.1"
}

With a few simple changes, the App's information is now being read from the app.conf file. The package.conf file couldn't be completely removed, as we need to store the Splunk version somewhere, to be shown on the About section of the App. That's the only use it has, actually.

image

AlexRuiz7 commented 2 years ago

The pre-release script

The generate-build-version.js pre-release script has been extended and updated to propagate the Wazuh version, App's revision and Splunk's supported minor version to the app.conf, package.conf and appVersionService.js files. This information is now centralized in the packgage.json file.

Notice how the script now propagates all the values properly:

splunk_versions

The metadata

The package.conf only contains the Splunk's supported minor version now. The Wazuh version and App's revision is stored on the app.conf file, and read from here.

The removal of the install_source_checksum property

Reading the documentation from Splunk about the app.conf, I've noticed that we had set the install_source_checksum property, which is not meant to be set by hand as Splunk populates this value automatically. For this reason, this value has been removed.

AlexRuiz7 commented 2 years ago

Improvement of the compatibility checks

Up to 4.3.0, the App implements a logic to check whether the backend and the frontend use the same version of the App.

To do this, the App reads and compares the target Wazuh version and the revision number of the App (from both components). There are two problems on this approach:

  1. The Wazuh version comparison is not required, as the App's revision is unique for each release. Comparing the App's revision from each component is enough.
  2. The Splunk version is not taken into account, so in practice, a package of the App meant for a different Splunk version could be installed without problems, not providing a single warning but allowing a potentially unstable installation.

Here's the comparison logic:

https://github.com/wazuh/wazuh-splunk/blob/d634cbc2ba1ba38f2a541dfc3ce1e737cb2a35e6/SplunkAppForWazuh/appserver/static/js/services/app-version/appVersionService.js#L42-L43

On this code, appVersion is the App's metadata from the frontend, stored as the header of the appVersionService.js file, and appPackageVersion is the metadata from the backend (seen on a previous comment). These names are clearly confusing, and should be renamed to something more descriptive, as ui_metadata and backend_metadata respectively. With the proposed changes, this comparison would look as follows:

        ui_metadata.revision !== backend_metadata.revision

On another note, the version of Splunk is currently not being compared, as mentioned above. Neither is the version of Wazuh API, but there are unused / outdated implemented routines for this.

https://github.com/wazuh/wazuh-splunk/blob/d634cbc2ba1ba38f2a541dfc3ce1e737cb2a35e6/SplunkAppForWazuh/appserver/static/js/services/api-manager/apiMgrService.js#L336-L369

Required work

AlexRuiz7 commented 2 years ago

Applied Changes

As a result of the work mentioned above, our App performs two kind of checks:

  1. Matching of the backend and frontend App's revision numbers.
    • Frequency: Checked once at App start.
    • Effect: Shows a warning.
  2. Matching of the App (frontend) and the Wazuh API versions.
    • Frequency: Checked once at App start.
    • Effect: Shows a warning.
  3. Matching of the App (backend) and the Wazuh API versions.
    • Frequency: Checked when a new API is attempted to be saved.
    • Effect: Shows an error and prevents the API from being added.

Missing work

I've found no way of querying the Splunk version on which the App is deployed. The only possibility I've found is to directly obtain it by using the splunk binary. Although this is feasible using Python, I don't think it's a good approach.