Simple Database Archival Solution (SDAS) is an open source solution which you can deploy in your AWS account to archive data to AWS. SDAS provides organizations with an efficient, easy and cost-effective solution for archiving Oracle, Microsoft SQL, PostgreSQL, and MySQL databases.
Apache License 2.0
39
stars
8
forks
source link
feat: Implement Async Processing with Lambda and DynamoDB for obtaining Source Tables #33
This commit introduces an asynchronous processing architecture using AWS Lambda and DynamoDB. The changes include:
Creation of ApiGetDatabaseTablesLambda, a Lambda function that handles initial API requests. It inserts job records into DynamoDB and triggers BackgroundGetSourceTablesLambda.
Setup of BackgroundGetSourceTablesLambda, a background Lambda function for long-running tasks. It processes data and updates job status in DynamoDB.
Addition of IAM roles with necessary permissions for both Lambdas, including DynamoDB read/write access and cross-Lambda invocation capabilities.
Configuration of DynamoDB table fetchTablesSchemaAsyncTable with TTL to manage automatic data expiry.
Update of API Gateway to route requests to ApiGetDatabaseTablesLambda.
This asynchronous design separates immediate response handling from intensive data processing, enhancing system scalability and responsiveness.
This commit introduces an asynchronous processing architecture using AWS Lambda and DynamoDB. The changes include:
ApiGetDatabaseTablesLambda, a Lambda function that handles initial API requests. It inserts job records into DynamoDB and triggersBackgroundGetSourceTablesLambda.BackgroundGetSourceTablesLambda, a background Lambda function for long-running tasks. It processes data and updates job status in DynamoDB.fetchTablesSchemaAsyncTablewith TTL to manage automatic data expiry.ApiGetDatabaseTablesLambda.This asynchronous design separates immediate response handling from intensive data processing, enhancing system scalability and responsiveness.