Trying to figure out how fig is supposed to know what url to connect to (security hub integration)

steven-tan commented 1 year ago

Hi there, I've been following instructions here:

https://www.crowdstrike.com/blog/tech-center/crowdstrike-aws-security-hub/ and https://github.com/CrowdStrike/Cloud-AWS/tree/main/Security-Hub

Cloudformation job seems to have ran okay. I'm able to log into the ec2 instance and confirm I have outbound connectivity, able to reach crowdstrike urls.

But fig.service is failing when I run. Looking at /var/log/messages, I see the following:

Jul 14 19:48:30 ip-10-0-0-157 python3: Failed to connect to the API on us1.  Check base_url and ssl_verify configuration settings.
Jul 14 19:48:30 ip-10-0-0-157 systemd: fig.service: main process exited, code=exited, status=1/FAILURE
Jul 14 19:48:30 ip-10-0-0-157 systemd: Unit fig.service entered failed state.
Jul 14 19:48:30 ip-10-0-0-157 systemd: fig.service failed.

The documentation confuses me a bit, because it talks about 6 parameters being needed, but the screenshot of Parameter Store values only shows 5 values (and does not include a base_url parameter. Looking directly at our AWS Parameter Store, I actually see there is no mention of any base_url parameter.

Digging around the code further here: https://github.com/CrowdStrike/Cloud-AWS/blob/main/Security-Hub/main.py

I see that if base_url isn't specified, it defaults to "us1" - which makes sense considering the error message output.

I'm wondering if someone can explain what went wrong, and if there's supposed to be some step about manually adding a particular URL entry to the parameter store and if so, how that should be formatted (I see two types of entries, for example both FIG_FALCON_CLIENT_ID and Falcon_ClientID parameters - and I want to make sure I know the correct parameter name to use, along with the proper value for the URL... I suspect it is: https://api.laggar.gcw.crowdstrike.com/ but it would be great to have confirmation.

jshcodes commented 1 year ago

That screenshot needs to be updated. The EC2 instance it should be looking at the FIG_API_BASE_URL parameter in Parameter Store for this value. (Defaulting to us1 / auto when it is not found.)

You are correct, you can use either the URL (with or without the https://) for this value, or you can use the shortname (usgov1).

steven-tan commented 1 year ago

Thanks for the clarification @jshcodes ! The error isn't happening in logs now, but now I get a different message... any idea what might be causing this?

[ec2-user@sechub-crowdstrike-integration-2023-q3-fig log]$ sudo service fig status
Redirecting to /bin/systemctl status fig.service
● fig.service - Security Hub Integration
   Loaded: loaded (/usr/lib/systemd/system/fig.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Fri 2023-07-14 21:31:28 UTC; 1min 21s ago
  Process: 7883 ExecStart=/usr/bin/python3 /usr/share/fig/main.py &> /dev/null (code=exited, status=0/SUCCESS)
 Main PID: 7883 (code=exited, status=0/SUCCESS)

Jul 14 21:31:28 sechub-crowdstrike-integration-2023-q3-fig systemd[1]: fig.service holdoff time over, scheduling restart.
Jul 14 21:31:28 sechub-crowdstrike-integration-2023-q3-fig systemd[1]: Stopped Security Hub Integration.
Jul 14 21:31:28 sechub-crowdstrike-integration-2023-q3-fig systemd[1]: start request repeated too quickly for fig.service
Jul 14 21:31:28 sechub-crowdstrike-integration-2023-q3-fig systemd[1]: Failed to start Security Hub Integration.
Jul 14 21:31:28 sechub-crowdstrike-integration-2023-q3-fig systemd[1]: Unit fig.service entered failed state.
Jul 14 21:31:28 sechub-crowdstrike-integration-2023-q3-fig systemd[1]: fig.service failed.

carlosmmatos commented 1 year ago

@steven-tan - Assuming you made the correct changes already to the FIG_API_BASE_URL, I would say to checkout the troubleshooting section, specifically around checking the application logs.

Also - running the application in standalone mode will help you better see the output for identifying potential issues.

Let us know what you find.

steven-tan commented 1 year ago

@carlosmmatos - thanks for the response.

The application log shows this repeatedly, I'm not really sure how much of this is problematic (in particular missing parameters indicated which aren't reflected in the docs I think):

[ec2-user@sechub-crowdstrike-integration-2023-q3-fig log]$ tail -25 /usr/share/fig/fig-service.log

Thu Jul 20 18:32:52 2023 Specified configuration file not found

Thu Jul 20 18:32:52 2023 FIG_FALCON_CLIENT_ID parameter loaded successfully.

Thu Jul 20 18:32:52 2023 FIG_FALCON_CLIENT_SECRET parameter loaded successfully.

Thu Jul 20 18:32:52 2023 FIG_APP_ID parameter loaded successfully.

Thu Jul 20 18:32:52 2023 FIG_SEVERITY_THRESHOLD parameter loaded successfully.

Thu Jul 20 18:32:52 2023 FIG_SQS_QUEUE_NAME parameter loaded successfully.

Thu Jul 20 18:32:52 2023 FIG_API_BASE_URL parameter loaded successfully.

Thu Jul 20 18:32:52 2023 FIG_CONFIRM_PROVIDER SSM parameter not found

Thu Jul 20 18:32:52 2023 FIG_SSL_VERIFY SSM parameter not found

Thu Jul 20 18:32:52 2023 Configuration parameters loaded from SSM Parameter Store.

Thu Jul 20 18:32:53 2023 No streams available

Thu Jul 20 18:32:53 2023 Process terminated

steven-tan commented 1 year ago

Also, running the command manually didn't yield anything I could see either...

[ec2-user@sechub-crowdstrike-integration-2023-q3-fig log]$ cd /usr/share/fig
[ec2-user@sechub-crowdstrike-integration-2023-q3-fig fig]$ sudo -u fig python3 main.py
/home/fig/.local/lib/python3.7/site-packages/boto3/compat.py:82: PythonDeprecationWarning: Boto3 will no longer support Python 3.7 starting December 13, 2023. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.8 or later. More information can be found here: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/
  warnings.warn(warning, PythonDeprecationWarning)
[ec2-user@sechub-crowdstrike-integration-2023-q3-fig fig]$

carlosmmatos commented 1 year ago

@steven-tan - since you are on usgov1 - have you put in a support request to have event streams enabled for your cid?

From the event streams docs:

Note: If your CrowdStrike cloud is US-GOV-1 and your CID doesn’t have event streams enabled, or if the status is unknown, contact Support for assistance.

I just want to make sure we're not missing anything.

steven-tan commented 1 year ago

I appreciate the quick response @carlosmmatos - I'm pretty new to using/implementing this stuff, so a couple quick questions:

What does CID refer to? Not familiar with the term in this context
Are you able to link the actual event streams docs you are referring to? May help me answer my own questions.

I do believe we are on gov cloud, so if I understand correctly, I need to open a support case with CrowdStrike to ensure event stream configuration is correct. (Am going through that process now)

carlosmmatos commented 1 year ago

Sure no problem:

CID is your Customer ID - every customer has this. If you were interested to see what your's was - you could log onto the console and go to the sensor downloads page.

Host setup and management -> Deploy -> Sensor downloads

This is the link to the event streams docs: ( you may have to change the link if you use another cloud ) https://falcon.crowdstrike.com/documentation/89/event-streams-apis

carlosmmatos commented 1 year ago

@steven-tan I'm going to close this since it's been past 60 days. If you have any other questions, please open up a new issue.

Thanks

CrowdStrike / Cloud-AWS

Trying to figure out how fig is supposed to know what url to connect to (security hub integration) #242