CiscoDevNet / cloud-security

Other
57 stars 82 forks source link

Splunk add-on error: Exiting due to exception #17

Open dwalllsq opened 2 years ago

dwalllsq commented 2 years ago

I have installed Cisco Cloud Security Umbrella Add-on for Splunk, v1.0.22 on a heavy forwarder and followed the Product Guide for setup. Logging is enabled in Cisco Umbrella Admin interface and provided AWS access key and ID are used. When enabling the Splunk addon, I get messages in the logs stating it connected to the S3 instance, then exiting due to exception. If I use the same AWS Access Key ID and AWS Access Key, I am able to successfully see the logs and pull them down to my heavy forwarder manually, but not through the add-on.

Splunk Addon: https://splunkbase.splunk.com/app/5557/#/details

Cisco Product Guide: https://github.com/CiscoDevNet/cloud-security/blob/master/Cisco%20Cloud%20Security/Splunk/CiscoCS%20%20Splunk%20App%20ProductGuide_V1.0.22.pdf

Working test: AWS_ACCESS_KEY_ID=<My AWS Access Key ID> AWS_SECRET_ACCESS_KEY=<My AWS Access Key> AWS_DEFAULT_REGION=<My AWS Region> aws s3 ls s3://cisco-managed-<My AWS Region>/<AWS Directory Prefix>/dnslogs/

               PRE 2022-05-31/
                   PRE 2022-06-01/

Using the same command on one of these directories provides a list of files, and using a BASH script I am able to use aws s3 sync to pull the files down.

Splunk addon messages:

DEBUG pid=29648 tid=MainThread file=client.py:_register_legacy_retries:159 | Registering retry handlers for service: s3
INFO pid=29648 tid=MainThread file=base_modinput.py:log_info:295 | Connected to S3 instance[Region=<My AWS Region>, Bucket=cisco-managed-<My AWS Region>, Directory Prefix=<AWS Directory Prefix>/dnslogs/]
INFO pid=29648 tid=MainThread file=base_modinput.py:log_info:295 | Exiting due to exception and next execution will be continued from the check point date (2022/05/31)
INFO pid=29648 tid=MainThread file=base_modinput.py:log_info:295 | Trigger Audit: [Name: umbrellaDNS, Files: 0, Lines: 0]
INFO pid=29648 tid=MainThread file=base_modinput.py:log_info:295 | Disconnected from S3 instance

No logs are downloading using the addon, and no helpful message is displayed to tell me what the exception is, even with DEBUG enabled.

olegunza commented 2 years ago

Could you try next thing:

Clone the current input
In the new input:
    Enter the secret - even though it seems populated - its not so needs to be re-entered
    Enter the start date as: 2022-06-01
tuxjockey commented 2 years ago

Tried the add-on version v1.0.22 and is experiencing the same issue which @dwalllsq reported. I tried the workaround mentioned by @olegunza and had no luck. Tried bumping the start dates as well.

image

dwalllsq commented 2 years ago

Could you try next thing:

Clone the current input
In the new input:
    Enter the secret - even though it seems populated - its not so needs to be re-entered
    Enter the start date as: 2022-06-01

I had tried inputting the secret, and even deleting the plugin, recreating the Cisco logging setup, and re-creating the plugin with the new settings. I just re-tried the secret again and it is not working.

The only messages I am seeing in the ciscocloudsecurity.log is:

INFO 2022-07-11 19:09:21,792 CiscoCloudSecurity : MI: cloudlock : execution started ERROR 2022-07-11 19:09:21,987 CiscoCloudSecurity : MI: cloudlock, Exception : No active cloudlock settings Traceback (most recent call last): File "/opt/splunk/etc/apps/cisco-cloud-security/bin/cloudlock.py", line 290, in stream_events raise Exception('No active cloudlock settings') Exception: No active cloudlock settings ERROR 2022-07-11 19:09:21,987 CiscoCloudSecurity : MI: cloudlock, Exception : No active cloudlock settings Traceback (most recent call last): File "/opt/splunk/etc/apps/cisco-cloud-security/bin/cloudlock.py", line 290, in stream_events raise Exception('No active cloudlock settings') Exception: No active cloudlock settings

and in ta_cisco_cloud_security_umbrella_addon_cisco_cloud_security_umbrella_addon.log: 2022-07-11 19:13:28,676 INFO pid=17145 tid=MainThread file=base_modinput.py:log_info:295 | Connected to S3 instance[Region=us-east-1, Bucket=cisco-managed-us-east-1, Directory Prefix=<PREFIX>/dnslogs/] 2022-07-11 19:13:28,678 INFO pid=17145 tid=MainThread file=base_modinput.py:log_info:295 | Exiting due to exception and next execution will be continued from the check point date (2022/04/22) 2022-07-11 19:13:28,678 INFO pid=17145 tid=MainThread file=base_modinput.py:log_info:295 | Trigger Audit: [Name: umbrellaDNS, Files: 0, Lines: 0] 2022-07-11 19:13:28,678 INFO pid=17145 tid=MainThread file=base_modinput.py:log_info:295 | Disconnected from S3 instance