aws-solutions / workload-discovery-on-aws

Workload Discovery on AWS is a solution to visualize AWS Cloud workloads. With it you can build, customize, and share architecture diagrams of your workloads based on live data from AWS. The solution maintains an inventory of the AWS resources across your accounts and regions, mapping their relationships and displaying them in the user interface.
https://aws.amazon.com/solutions/implementations/workload-discovery-on-aws/
Apache License 2.0
727 stars 88 forks source link

Regarding Architecture issue #498

Closed SHRIDHARMUDASHI closed 10 months ago

SHRIDHARMUDASHI commented 10 months ago

Hi Team,

I have built the AWS Workload Discovery as per the template. However, I am unable to generate the entire architecture in diagrams, and I cannot see any services such as S3, EC2, and RDS in the resources. Please help me with this issue.

Thanks Shridhar

svozza commented 10 months ago

This is very likely a configuration issue. I have created a flow chart to help diagnose common issues when deploying the solution:

https://github.com/aws-solutions/workload-discovery-on-aws/blob/main/debugging-flowchart.mmd

SHRIDHARMUDASHI commented 10 months ago

Thanks for the details I have checked everything looks ok but still I am facing the issue

svozza commented 10 months ago

Check the ECS logs as described here, if there are no errors there then there most likely is something going on with how Config has been enabled:

https://aws-solutions.github.io/workload-discovery-on-aws/workload-discovery-on-aws/2.0/debugging-the-discovery-component.html

Also, check the that there are no errors in the CloudFormation console when you deployed the global and regional templates in any of the accounts you imported

SHRIDHARMUDASHI commented 10 months ago

Checked but no luck

svozza commented 10 months ago

When you say no luck, do you mean there were no errors in the ECS logs or the CloudFormation deployments?

SHRIDHARMUDASHI commented 10 months ago

Yes no error logs ECS and Cloudformation

SHRIDHARMUDASHI commented 10 months ago

Why is the Service policy displayed in resources image

SHRIDHARMUDASHI commented 10 months ago

Cloudformation side no issue All are deployed properly.

svozza commented 10 months ago

Those policies are there because we get them using the AWSD SDK not AWS Config (we get about 90% of resources from Config). The resources that are missing for you all come from Config. Go to one of the accounts you've imported and then pick one of the regions you've imported and check Authorization page on Config console. You should see the account that you deployed WD to saying that it's authorised.

Screenshot 2024-01-12 at 11 31 13
SHRIDHARMUDASHI commented 10 months ago

log-events-viewer-result.csv

svozza commented 10 months ago

{"message":"0 resources downloaded from Config advanced query","level":"info","timestamp":"2024-01-12T11:46:15.208Z"}

There is nothing in your Config aggregator. The account must not be authorized to send data to the WD aggregator.

SHRIDHARMUDASHI commented 10 months ago

image

SHRIDHARMUDASHI commented 10 months ago

Can you please quickly guide me on how to config?

svozza commented 10 months ago

Can you show me the screen from the WD aggregator, it will have a name similar to the one in this scrrenshot. Note how my aggregator has 6408 resources.

Screenshot 2024-01-12 at 12 07 01
SHRIDHARMUDASHI commented 10 months ago

image

SHRIDHARMUDASHI commented 10 months ago

Why Resource Inventory is empty?

svozza commented 10 months ago

I don't know why it's zero, this is an issue with Config, not Workload Discovery, it could be any number of things. When you click on the aggregator link on that page, do you see any errors like the one in this screenshot. If there are and you click on the red FAILED message it will give you an error message.

Screenshot 2024-01-12 at 12 19 42
SHRIDHARMUDASHI commented 10 months ago

image

svozza commented 10 months ago

Was Config already set up in this particular account?

SHRIDHARMUDASHI commented 10 months ago

No we have not done anything.

svozza commented 10 months ago

Was it set up in the account you were trying to import?

SHRIDHARMUDASHI commented 10 months ago

Yes

svozza commented 10 months ago

So just to clarify, the account you deployed WD to did not have Config enable but the account you are trying to import did have have Config enabled? In the account you are trying to import (i.e., not the account that WD is deployed to) can you check the Config settings and see if there's anything unusual in there?

Screenshot 2024-01-12 at 12 33 23
SHRIDHARMUDASHI commented 10 months ago

image

SHRIDHARMUDASHI commented 10 months ago

image

svozza commented 10 months ago

Config is not set up properly. Did you de[ploy the regional template in this account with the AlreadyHaveConfigSetup parameter set to Yes? It appears to me that Config was not enabled in this region. If you update the CFN stack that deployed the regional stack and change that parameter to No, it will set up the delivery channel for you.

SHRIDHARMUDASHI commented 10 months ago

Let me check

SHRIDHARMUDASHI commented 10 months ago

image

Getting this error

SHRIDHARMUDASHI commented 10 months ago

image

svozza commented 10 months ago

I would just start from scratch with the regional stack. Find the ConfigBucket in the S3 console and empty it and then delete the bucket. Once that is complete, delete the regional template CFN stack completely and then redeploy with AlreadyHaveConfigSetup set to No.

SHRIDHARMUDASHI commented 10 months ago

OK Let me try

SHRIDHARMUDASHI commented 10 months ago

image

svozza commented 10 months ago

Config has obviously got itself into a broken state. Delete the recorder and delivery channel manually using the CLI and then try to redeploy the regional resources stack.

# delete the recorder first
aws configservice delete-configuration-recorder --configuration-recorder-name default

# delete the delivery channel
aws configservice delete-delivery-channel --delivery-channel-name default
SHRIDHARMUDASHI commented 10 months ago

OK

SHRIDHARMUDASHI commented 10 months ago

Finally image

svozza commented 10 months ago

It will take a few minutes for the resources in Config to be discovered and then replicated to the WD config aggregator.

SHRIDHARMUDASHI commented 10 months ago

OK Thanks

SHRIDHARMUDASHI commented 10 months ago

image

svozza commented 10 months ago

Check if those resources are available in the WD config aggregator. The WD discovery process runs every 15 minutes so it will next run in 11 minutes. You should see resources n the UI after that.

SHRIDHARMUDASHI commented 10 months ago

Ok Thanks

SHRIDHARMUDASHI commented 10 months ago

All good thank you so much.

svozza commented 10 months ago

Great news!!