search

awslabs / backstage-plugins-for-aws

AWS plugins for Backstage
Apache License 2.0
66 stars 11 forks source link

feature: Ingest AWS resources in to the Backstage catalog #177

Open mrod23 opened 3 months ago

mrod23 commented 3 months ago

🔖 Feature description

I'd like to scrape AWS accounts for resources and then add as components to Backstage. Example: An RDS instance has the tag Key: BackstageService, Value: TestDB it would be added as a component to Backstage.

Is this already available?

🎤 Context

I'm looking at a competing product called Cortex and it looks like this feature is already available. https://docs.cortex.io/docs/reference/integrations/aws

✌️ Possible Implementation

No response

niallthomson commented 2 months ago

Hi @mrod23 thanks for raising this issue. We have explored this before and have some prototypes of creating Resource kind entities. However we're still figuring out requirements that are clear enough to implement it.

For example we wrote an entity provider that would create something like this for an RDS cluster:

apiVersion: backstage.io/v1alpha1
kind: Resource
metadata:
  namespace: default
  annotations:
    aws.amazon.com/arn: arn:aws:rds:us-west-2:1234567890:cluster:my-app-db
    aws.amazon.com/resource-type: AWS::RDS::DBCluster
    aws.amazon.com/name: my-app-db
    aws.amazon.com/region: us-west-2
  name: my-app-db
  description: AWS Config Resource AWS::RDS::DBCluster my-app-db
spec:
  type: rds-dbcluster

You'll note it only has generic information and no specific fields about the RDS database itself (engine version etc). This was intentional for our first pass. We know folks will likely want certain fields propagated to the entity, for example to potentially use Tech Insights or Soundcheck to perform checks. However I'm not sure we want to just dump the entire AWS resource information in the Backstage catalog.

Can you share details on your use-case here? Is it mainly to understand dependencies and relationships or are you looking for more information associated with the entities for checks?

mrod23 commented 2 months ago

My initial use-case is to understand dependencies and relationships.

Eventually for things like RDS, we would like more metadata such as engine version. This would be used by security team to understand which systems need updating.