awslabs / emr-dynamodb-connector

Implementations of open source Apache Hadoop/Hive interfaces which allow for ingesting data from Amazon DynamoDB
Apache License 2.0
216 stars 135 forks source link

Upgrade Hadoop/Hive versions and fix UT #169

Closed kevnzhao closed 1 year ago

kevnzhao commented 1 year ago

Issue #, if available:

Description of changes:

Upgrade hadoop.version and hive3.version so that all UTs can pass.


-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.dynamodb.write.WriteIopsCalculatorTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.594 sec
Running org.apache.hadoop.dynamodb.split.DynamoDBSplitGeneratorTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.081 sec
Running org.apache.hadoop.dynamodb.DynamoDBUtilTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.288 sec
Running org.apache.hadoop.dynamodb.test.GsonTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.07 sec
Running org.apache.hadoop.dynamodb.DynamoDBFibonacciRetryerTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.383 sec
Running org.apache.hadoop.dynamodb.util.ClusterTopologyNodeCapacityProviderTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.085 sec
Running org.apache.hadoop.dynamodb.util.RoundRobinYarnContainerAllocatorTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
Running org.apache.hadoop.dynamodb.util.TaskCalculatorTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec
Running org.apache.hadoop.dynamodb.DynamoDBClientTest
Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 141.172 sec
Running org.apache.hadoop.dynamodb.read.DynamoDBRecordReaderTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.184 sec
Running org.apache.hadoop.dynamodb.read.ReadIopsCalculatorTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 sec
Running org.apache.hadoop.dynamodb.read.PageResultMultiplexerTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.458 sec
Running org.apache.hadoop.dynamodb.preader.RateControllerTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.265 sec
Running org.apache.hadoop.dynamodb.preader.TokenBucketTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec
Running org.apache.hadoop.dynamodb.preader.ReadManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.535 sec
Running org.apache.hadoop.dynamodb.preader.ScanRecordReadRequestTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.061 sec
Running org.apache.hadoop.dynamodb.DynamoDBItemWritableTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.163 sec

Results :

Tests run: 80, Failures: 0, Errors: 0, Skipped: 0

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.hive.dynamodb.util.HiveDynamoDBUtilTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.933 sec
Running org.apache.hadoop.hive.dynamodb.type.HiveDynamoDBTypeTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.179 sec
Running org.apache.hadoop.hive.dynamodb.DynamoDBObjectInspectorTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
Running org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandlerTest
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.06 sec
Running org.apache.hadoop.hive.dynamodb.filter.DynamoDBFilterPushdownTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.479 sec
Running org.apache.hadoop.hive.dynamodb.DynamoDBSerDeTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec

Results :

Tests run: 57, Failures: 0, Errors: 0, Skipped: 0

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.dynamodb.exportformat.ExportOutputFormatTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.988 sec
Running org.apache.hadoop.dynamodb.exportformat.ExportManifestEntryTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.036 sec
Running org.apache.hadoop.dynamodb.exportformat.ExportFileFlusherTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.373 sec

Results :

Tests run: 11, Failures: 0, Errors: 0, Skipped: 0

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.