We are working on Dynamodb Cross Region Replication using latest CRR library(1.2.1). We have replicated almost 100 tables from Ireland to Oregon by running replication jar on Autoscaled EC2 Instances.
The current setup of Cross Region Replication is on the EC2 instances that are configured periodically to post memory utilization data of the AutoScaling Group to the AWS CloudWatch which is done using custom monitoring script provided by AWS. This data is being used to raise alarms which triggers the auto scaling policies to add/remove an instance based on the memory utilization threshold.
We are using CloudWatch service to raise alarms based on memory utilization statistics provided by AWS CloudWatch custom metrics and monitoring scripts.
We are using CRR jar 1.2.1 provided by AWS over GitHub for our current solution of cross region replication, which is responsible for managing streams for data replication internally on its own. It is utilizing the IAM user dynamodbCrossReplication, and polling that PutMetricData API on CloudWatch.
We want to reduce the Cloudwatch charges which is being caused due to the PutMetricData API calls made by the IAM user: dynamodbCrossReplication.
We also will like to tell you that the autoscaled instances are configured on the basis of memory utilization statistics using AWS CloudWatch custom metrics and monitoring scripts.
As suggested by source, it could be done easily by disabiling the Cloudwatch metrics, by passing the argument --dontPublishCloudwatch flag in the jar command, as mentioned below,
java -jar dynamodb-cross-region-replication-1.2.1.jar --sourceRegion --sourceTable --destinationRegion --destinationTable --dontPublishCloudwatch
The number of records and bytes processed will not get recorded for them, when the jar will be executed.
2) Basically it looks like we are polling a specific metric at a very high frequency and would need to reduce the polling frequency, which can be done by reducing the value of DEFAULT_PARENT_SHARD_POLL_INTERVAL_MILLIS in DynamoDBConnectorConstants.java from 10000L to lower the frequency rate value.
Would any of the above mentioned way, effect the running replication processes in any way?
Please help us with the best solution out of these two or if there is any, to reduce the cloudwatch charge which does not effect our current setup of replication adversely.
We are working on Dynamodb Cross Region Replication using latest CRR library(1.2.1). We have replicated almost 100 tables from Ireland to Oregon by running replication jar on Autoscaled EC2 Instances. The current setup of Cross Region Replication is on the EC2 instances that are configured periodically to post memory utilization data of the AutoScaling Group to the AWS CloudWatch which is done using custom monitoring script provided by AWS. This data is being used to raise alarms which triggers the auto scaling policies to add/remove an instance based on the memory utilization threshold. We are using CloudWatch service to raise alarms based on memory utilization statistics provided by AWS CloudWatch custom metrics and monitoring scripts. We are using CRR jar 1.2.1 provided by AWS over GitHub for our current solution of cross region replication, which is responsible for managing streams for data replication internally on its own. It is utilizing the IAM user dynamodbCrossReplication, and polling that PutMetricData API on CloudWatch. We want to reduce the Cloudwatch charges which is being caused due to the PutMetricData API calls made by the IAM user: dynamodbCrossReplication. We also will like to tell you that the autoscaled instances are configured on the basis of memory utilization statistics using AWS CloudWatch custom metrics and monitoring scripts. As suggested by source, it could be done easily by disabiling the Cloudwatch metrics, by passing the argument --dontPublishCloudwatch flag in the jar command, as mentioned below, java -jar dynamodb-cross-region-replication-1.2.1.jar --sourceRegion --sourceTable --destinationRegion --destinationTable --dontPublishCloudwatch
The number of records and bytes processed will not get recorded for them, when the jar will be executed.
2) Basically it looks like we are polling a specific metric at a very high frequency and would need to reduce the polling frequency, which can be done by reducing the value of DEFAULT_PARENT_SHARD_POLL_INTERVAL_MILLIS in DynamoDBConnectorConstants.java from 10000L to lower the frequency rate value.
Would any of the above mentioned way, effect the running replication processes in any way?
Please help us with the best solution out of these two or if there is any, to reduce the cloudwatch charge which does not effect our current setup of replication adversely.