Have been facing this issue since i installed the scoutsuite, its just stuck on SECRETS MANAGER SERVICE

[BluefrostOnline]

I just created this account and trying to raise a issue, Installed scoutsuite , configured the user as well inside aws cli. its running and then gets stuck on "Scout[209963] INFO Fetching resources for the Secrets Manager service" Can anyone help here quickly?

[x4v13r64]

It's not stuck it's just making API calls which take some time to complete.

[nickdnk]

I have this problem. It reaches this point, then my network connection starts bugging out (Shania Twain stops singing off Spotify) and then it errors out saying it could not resolve the endpoints. I tried setting the ulimit -Sn 1000, but it does the same thing. Lots of lines like this:

ec2.py L90: Failed to describe EC2 VPC: Could not connect to the endpoint URL: "https://ec2.us-east-1.amazonaws.com/"
2022-08-16 16:41:03 macbook-pro.local scout[34912] ERROR ec2.py L115: Failed to get EC2 volumes: Could not connect to the endpoint URL: "https://ec2.us-east-2.amazonaws.com/"

Following that, all the subsequent requests fail. It happens at "Secrets Manager Service" every time.

This is on macOS 12.5

What's going on here?

[fernando-gallego]

Hi @nickdnk , can you still reproduce this issue with the new version 5.12.0? If so please paste error output here.

[panda89p13]

@fernando-gallego I have the same issue, hanging at secrets manager i.e making apis calls forever without response

[a-ortuno]

@fernando-gallego I am also having the same problem. This is on a Mac 12.3.

On the first pass of the script on AWS it got stuck on secret manager and then inmediately starts throwing errors like for a bunch of AWS services:

sns.py L15: Failed to get CloudWatch alarms: Could not connect to the endpoint URL: "https://sns.ap-northeast-2.amazonaws.com/"

On the second try of the tool, I got the following errors for a bunch of AWS services:

ERROR cloudformation.py L30: Failed to describe CloudFormation stack: An error occurred (Throttling) when calling the DescribeStacks operation (reached max retries: 4): Rate exceeded

Please let me know if you need more details. Thanks

Edit: Reducing the number of threats improves the situation of errors with --max-workers 5

[hemanth-codaio]

We've been seeing this recently with 5.12.0 as well. ~Using AWS_RETRY_MODE=adaptive and --max-rate isn't really helping.~ Curious if there's been changes to the tool that are causing this rate limit to get hit way more frequently?

Edit: I was able to get Scoutsuite working 100% of the time (albeit with a doubling of runtime) with --max-rate 5, --max-workers 5, and AWS_RETRY_MODE=adaptive, in case it helps anyone else!

[pak0s]

I was able to get Scoutsuite working 100% of the time (albeit with a doubling of runtime) with --max-rate 5, --max-workers 5, and AWS_RETRY_MODE=adaptive, in case it helps anyone else!

This did not help me. Using these 3 arguments slowed down ScoutSuite but it still eventually failed after a long time.

[JacobWilson01]

@fernadno-gallego

Based on the comments above, this is really a rate limit issue within AWS. Either it runs too fast and there are errors, or it runs too slow and takes forever. I am trying to get this completed with a 100% success rate within a lambda, and it is not possible with 5.12.0. Setting max_rate, max_workers, and AWS_RETRY_MODE to adaptive improves the success rate, but it never always works. The best I can do is successfully running a little over 70%, which can be frustrating. Are we able to get a solution for the follow issues:

https://github.com/nccgroup/ScoutSuite/issues/666#issue-579401551 https://github.com/nccgroup/ScoutSuite/issues/1396#issue-1113991881

[fernando-gallego]

Hi, if ScoutSuite reports an error similar to 'Could not connect to the endpoint URL: "https://sns.ap-northeast-2.amazonaws.com/"', the issue is not in the tool itself but rather on your network. We have seen similar errors in the past occurring when ScoutSuite is launched in restricted networks, or when ScoutSuite traffic has to traverse network elements such as firewalls, which may be blocking traffic destined to specific regions.

If you encounter 'Could not connect to the endpoint URL' errors, check if there is any throttling/traffic filtering happening on your network. The endpoint URLs are correct as per AWS documentation.

[maxammann]

I'm relatively sure the issue is that scoutsuite opens too many files and does not close them. Opening sockets also counts towards this limit.

So simply increase the limit on your OS.

This is an error I got:

2023-06-01 11:06:19 max-mac scout[40820] INFO Opening the HTML report
Traceback (most recent call last):
...
OSError: [Errno 24] Too many open files
AWS_RETRY_MODE=adaptive scout aws --max-rate 5 --max-workers 5  96.37s user 6.55s system 31% cpu 5:28.16 total

[christolourens01]

I'm having the same issue, Mac Ventura 13.4. I've tried these fixes including updating the amount of sockets available and it still just stalls out at Fetching resources for the Secrets Manager service.