search

splunk / splunk-add-on-microsoft-azure

Splunk Add-on for Microsoft Azure
Apache License 2.0
11 stars 7 forks source link

Enums module error #38

Closed akearney-sky closed 1 year ago

akearney-sky commented 1 year ago

After installing TA-MS-AAD v4.0.2 (Splunk 8.2.9) I am getting a lot of errors generated that are of the following form:

Traceback (most recent call last):
  File "/apps/splunk/splunk/etc/apps/TA-MS-AAD/bin/azure_virtual_network.py", line 23, in <module>
    import json
  File "/apps/splunk/splunk/lib/python3.7/json/__init__.py", line 106, in <module>
    from .decoder import JSONDecoder, JSONDecodeError
  File "/apps/splunk/splunk/lib/python3.7/json/decoder.py", line 3, in <module>
    import re
  File "/apps/splunk/splunk/lib/python3.7/re.py", line 145, in <module>
    class RegexFlag(enum.IntFlag):
AttributeError: module 'enum' has no attribute 'IntFlag'

These are seen originating from many different TA-MS-AAD/bin/*.py files.

Interestingly these errors are only being generated on the Search Head Cluster, not on the Heavy Forwarder.

I have tried digging a bit further by don't have python knowledge to debug any further. Some things that I have found:

It has something to with the wrong enum module being loaded when running on the SHC (possibly something to do with enum34??

In the Splunk directory on both a SH and HF I can see many different versions of enums.py. In SPLUNK/lib/python*/ , in the TA-MS-AAD app folder and in other app folders. I am unsure how to know which one is being used.

On the HF there is one input where "python.version = python2", though all others have "python.version = python3"

akearney-sky commented 1 year ago

It turned out that TA-MS-AAD had been upgraded from an older version. This was performed by deleting the old app directory on the Deployer (while backing up local), then copying in the directory for the new version of the app (then adding in local changes). The app was then deployed to the search head cluster members (splunk apply shcluster-bundle).

It turns out that this doesn't cause files that are no longer present in the new app directory (e.g. the *.py files) to be deleted from the Search Head Cluster members (see: https://community.splunk.com/t5/Deployment-Architecture/Applying-search-head-cluster-bundle-not-removing-old-directories/m-p/261332).

To work around this I had to remove the old directory entirely from the deployer and deploy the cluster bundle (this removed the app entirely from the SH cluster members). Then copy the new version of the app to the deployer and deploy the cluster bundle again.