search

audienceproject / spark-dynamodb

Plug-and-play implementation of an Apache Spark custom data source for AWS DynamoDB.
Apache License 2.0
175 stars 90 forks source link

IllegalArgumentException: No region provided #81

Open franciriva opened 4 years ago

franciriva commented 4 years ago

Hi there, I'm trying to upgrade to the latest version: spark-dynamodb_2.12-1.1.0 but no matter what I do I get the above error: "IllegalArgumentException: No region provided"

The code works on this version: spark-dynamodb_2.11-1.0.4

Code here:

df = spark.read.schema(source_raw_schema).format("dynamodb") \
  .option("region", "eu-west-1") \
  .option("roleArn", "arn:aws:iam::xxx") \
  .option("tableName", "yyy") \
  .load()

Can you help?

Thanks!

youcefjd commented 4 years ago

Hi - not related to your issue and I apologize for that. I was just wondering if you could help me out to understand how to load data from dynamo using the AP library? Can you provide me with a code snippet on how to connect dynamo to spark and then load the data to a df? Thanks