search

brianfrankcooper / YCSB

Yahoo! Cloud Serving Benchmark
Apache License 2.0
4.93k stars 2.23k forks source link

Question: Cannot run YCSB for Dynamo Client #989

Closed anirban-roy closed 7 years ago

anirban-roy commented 7 years ago

Hello, This is a question. Any quick re I am trying to run YCSB for DynamoDB (provisioned for 100 RCU and 50 WCUs). Following is the command I am executing:

./bin/ycsb load dynamodb -P dynamodb-binding/conf/dynamodb.properties -P workloads/workloada -threads 1 -target 40

my properties has the endpoint (us-east-1), aws credentials etc. defined. I can run the shell with inserts: ./bin/ycsb shell dynamo

The table schema has only 1 field named: partition_key but when I try to perform a load I get the following errors: ./bin/ycsb load dynamodb -P dynamodb-binding/conf/dynamodb.properties -P workloads/workloada -threads 1 -target 40

java -cp /opt/ycsb-0.12.0/dynamodb-binding/conf:/opt/ycsb-0.12.0/conf:/opt/ycsb-0.12.0/lib/core-0.12.0.jar:/opt/ycsb-0.12.0/lib/HdrHistogram-2.1.4.jar:/opt/ycsb-0.12.0/lib/htrace-core4-4.1.0-incubating.jar:/opt/ycsb-0.12.0/lib/jackson-core-asl-1.9.4.jar:/opt/ycsb-0.12.0/lib/jackson-mapper-asl-1.9.4.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-api-gateway-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-autoscaling-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cloudformation-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cloudfront-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cloudhsm-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cloudsearch-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cloudtrail-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cloudwatch-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cloudwatchmetrics-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-codecommit-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-codedeploy-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-codepipeline-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cognitoidentity-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-cognitosync-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-config-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-core-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-datapipeline-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-devicefarm-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-directconnect-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-directory-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-dynamodb-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-ec2-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-ecr-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-ecs-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-efs-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-elasticache-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-elasticbeanstalk-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-elasticloadbalancing-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-elasticsearch-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-elastictranscoder-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-emr-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-events-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-glacier-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-iam-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-importexport-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-inspector-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-iot-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-kinesis-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-kms-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-lambda-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-logs-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-machinelearning-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-marketplacecommerceanalytics-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-opsworks-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-rds-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-redshift-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-route53-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-s3-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-ses-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-simpledb-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-simpleworkflow-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-sns-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-sqs-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-ssm-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-storagegateway-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-sts-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-support-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-swf-libraries-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-waf-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/aws-java-sdk-workspaces-1.10.48.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/commons-codec-1.6.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/commons-logging-1.1.3.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/dynamodb-binding-0.12.0.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/httpclient-4.3.6.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/httpcore-4.3.3.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/jackson-annotations-2.5.0.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/jackson-core-2.5.3.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/jackson-databind-2.5.3.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/joda-time-2.8.1.jar:/opt/ycsb-0.12.0/dynamodb-binding/lib/log4j-1.2.17.jar com.yahoo.ycsb.Client -db com.yahoo.ycsb.db.DynamoDBClient -P dynamodb-binding/conf/dynamodb.properties -P workloads/workloada -threads 1 -target 40 -load YCSB Client 0.12.0 Command line: -db com.yahoo.ycsb.db.DynamoDBClient -P dynamodb-binding/conf/dynamodb.properties -P workloads/workloada -threads 1 -target 40 -load Loading workload... Starting test. 0 [Thread-1] INFO com.yahoo.ycsb.db.DynamoDBClient -dynamodb connection created with http://dynamodb.us-east-1.amazonaws.com DBWrapper: report latency for each error is false and specific error codes to track for latency are: [] 435 [Thread-1] ERROR com.yahoo.ycsb.db.DynamoDBClient -com.amazonaws.AmazonServiceException: One or more parameter values were invalid: Missing the key partition_key in the item (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: BOJ5PRDH5N2H40TDH04ERN47BBVV4KQNSO5AEMVJF66Q9ASUAAJG) Error inserting, not retrying any more. number of attempts: 1Insertion Retry Limit: 0 [OVERALL], RunTime(ms), 934.0 [OVERALL], Throughput(ops/sec), 0.0 [TOTAL_GCS_PS_Scavenge], Count, 1.0 [TOTAL_GC_TIME_PS_Scavenge], Time(ms), 8.0 [TOTAL_GCTIME%_PS_Scavenge], Time(%), 0.8565310492505354 [TOTAL_GCS_PS_MarkSweep], Count, 0.0 [TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0.0 [TOTAL_GCTIME%_PS_MarkSweep], Time(%), 0.0 [TOTAL_GCs], Count, 1.0 [TOTAL_GC_TIME], Time(ms), 8.0 [TOTAL_GCTIME%], Time(%), 0.8565310492505354 [CLEANUP], Operations, 1.0 [CLEANUP], AverageLatency(us), 1.0 [CLEANUP], MinLatency(us), 1.0 [CLEANUP], MaxLatency(us), 1.0 [CLEANUP], 95thPercentileLatency(us), 1.0 [CLEANUP], 99thPercentileLatency(us), 1.0 [INSERT], Operations, 0.0 [INSERT], AverageLatency(us), NaN [INSERT], MinLatency(us), 9.223372036854776E18 [INSERT], MaxLatency(us), 0.0 [INSERT], 95thPercentileLatency(us), 0.0 [INSERT], 99thPercentileLatency(us), 0.0 [INSERT], Return=ERROR, 1 [INSERT-FAILED], Operations, 1.0 [INSERT-FAILED], AverageLatency(us), 428928.0 [INSERT-FAILED], MinLatency(us), 428800.0 [INSERT-FAILED], MaxLatency(us), 429055.0 [INSERT-FAILED], 95thPercentileLatency(us), 429055.0 [INSERT-FAILED], 99thPercentileLatency(us), 429055.0

Could anyone please guide to understand as to what I am missing?

~Thanks

anirban-roy commented 7 years ago

Never mind!