Support for DynamoDB Document API

[ricardojmendez]

DynamoDB's Document API provides support for functionality that isn't on their base API, like requesting a maxResultSize on queries or scans. Are there any plans to support it?

You can read more here: http://java.awsblog.com/post/Tx1DDAWQGXWITSG/Introducing-DynamoDB-Document-API-Part-1 https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/dynamodbv2/document/DynamoDB.html

[mcohen01]

I don't think it would be too tedious to add support for the Document Api, but personally I have no plans to implement it. PRs welcome though :-)

[ricardojmendez]

I'm new to Amazonica - any guidelines? Do you use git-flow? For your approach, should I be looking beyond your uses of amazonica.core/set-client and intern-function and (say) the s3 and dymamodbv2 namespaces as examples?

[mcohen01]

Well, I just looked at the Document Api for a minutes and, to be honest, it might be rather difficult to get a whole lot of leverage using amazonica to help implement it. Amazonica relies on the fact that 99% of the AWS Java SDK follows the pattern of a single top level "client" class per service which contains all the behavior (i.e. functions) required, and then you "configure" each function call via standard Javabean calls of arbitrarily nested depth. So the library can use reflection to convert your clojure data structures into Javabean setter invocations, and call the getters to give you clojure data structures in return.

The Document Api does not follow that pattern for the most part. The main entry point is the DynamoDB class, which at first looks like it follows this pattern, the way the EC2Client does for instance, with a number of top level methods. But really, it looks like getTable, batchGetItem, and batchWriteItem are the useful ones, in particular getTable.

So quite often someone from the community will send a PR adding support for a new AWS service simply by adding a file like so. This is basically how amazonica works, you point it at a class file and it will convert every method of the class to idiomatically named clojure vars. So if you were to start with DynamoDB.class, only the listTables method would work. getTable would fail miserably, because if you called it, it would then reflectively try to invoke every get* method on the resulting Table object. (And actually, even listTables would be crippled, because the pages method would be unavailable to you.) But the meat of the api looks to be contained within the Table class. Forgetting the batch* methods of DynamoDB, you would get a lot of leverage if you used amazonica to generate a clojure api for Table. At first glance you would need to solve a few issues:

• you'd need to construct an AmazonDynamoDBClient to pass into the constructor.
• where does the table name constructor arg go?
• you'd need to handle the pages method (e.g. as returned by calls to scan)
• you'd need to modify the reflection code to look for add* methods and with* methods, as the Document Api uses more of a fluent api than Javabeans conventions - some parameters have no corresponding setters

I'm sure there would be other gotchas as well. It's a toss up, you would still get a fair amount of leverage by using amazonica, but there looks to be a nontrivial amount of work involved to make it a decent api. If you wanted to start on it, I guess I would create a new namespace for document, make amazon-client public, and see how functional Table is with no modifications.

[ricardojmendez]

Thanks for the detailed reply, I'll look into it. Cheers!