search

aws-solutions / workload-discovery-on-aws

Workload Discovery on AWS is a solution to visualize AWS Cloud workloads. With it you can build, customize, and share architecture diagrams of your workloads based on live data from AWS. The solution maintains an inventory of the AWS resources across your accounts and regions, mapping their relationships and displaying them in the user interface.
https://aws.amazon.com/solutions/implementations/workload-discovery-on-aws/
Apache License 2.0
729 stars 89 forks source link

Request for Comment: Displaying Networking Relationships #543

Open ConnorKirk opened 2 months ago

ConnorKirk commented 2 months ago

Summary

We're often asked if Workload Discovery can display networking information such as VPC flow logs as relationships between resources.

For example, "Can I see if this EC2 instance is communicating with this NAT Gateway".

Today, Workload Discovery presents configuration based relationships for networking resources, such as "EC2 instance X is contained in Subnet Y". Workload Discovery cannot show relationships between two resources that are communicating via a network. It might be possible to display this information in Workload Discovery. There may be other related information that is also useful to include.

We'd like to investigate what usecases users have for visualising relationships between networked resources in the tool.

We'd welcome any anecdotes or feedback on how we can improve Workload Discovery in this area. We will update this issue with more information as the investigation progresses.

What is the problem?

Customers would like to see relationships in WD representing actual networking communication between resources. Today, Workload Discovery only shows configuration based relationships such as EC2 instance being associated with an Network Interface. Workload Discovery does not show relationships derived from potential or actual network communication.

What is the solution?

Workload Discovery will support a new relationship type, a network derived relationship. Network derived relationships will be derived from VPC Flow Logs.

brianok-aws commented 2 months ago

I think this would be a fantastic addition to introduce new FinOps processes if added to the graph. I cover some of these techniques in graph at https://aws.amazon.com/blogs/database/techniques-to-improve-the-state-of-the-art-in-cloud-finops-using-amazon-neptune/, but adding the VPC Flows would also allow you to detect resources that have unused pathways in addition to no pathway. It will be interesting to see how you can enable these flows on existing resources with minimal disruption, as well as without significant cost. Also, I think it is important to differentiate between when monitoring is active and there is no traffic and when there is no monitoring...so techniques can be applied to the former but not the latter.

svozza commented 2 months ago

Also, I think it is important to differentiate between when monitoring is active and there is no traffic and when there is no monitoring...so techniques can be applied to the former but not the latter.

This is a very good point, we need to make this distinction clear in any diagrams.

NickB118 commented 1 month ago

I think this would be an excellent enhancement. We are looking at tools that do just this currently, specifically https://faddom.com/, but we have also tried a homegrown approach analysing VPC flow logs and outputting as graphviz previously. VPC flow log analysis has been of huge help to our team previously in getting to grips with understanding how legacy systems work and what is safe to decommission, but integration into Workload Discovery sounds like it could be of great use too.