aws / aws-sdk-ruby-record

Official repository for the aws-record gem, an abstraction for Amazon DynamoDB.
Apache License 2.0
318 stars 41 forks source link

Support for BatchGetItem #122

Closed yyamanoi1222 closed 1 year ago

yyamanoi1222 commented 2 years ago

Are there any plans to implement BatchGetItem in this gem?

I saw this issue https://github.com/aws/aws-sdk-ruby-record/issues/12 We have a requirement and BatchGetItem may be better than TransactGetItems. (e.g., retrieve large number of items , no need to consider transaction conflict errors, etc.)

If you don't plan to implement it and are be open the idea, may I add the feature myself? The API interface will be something like transact_find

result = Aws::Record::Batch.find(
  batch_items: [
    TableOne.bfind_opts(key: { uuid: "uuid1234" }),
    TableTwo.tfind_opts(key: { hk: "hk1", rk: "rk1"}), 
    TableTwo.tfind_opts(key: { hk: "hk2", rk: "rk2"})  
  ]
)

result.responses # { TableOne.table_name => [TableOne], TableTwo.table_name => [TableTwo, TableTwo] }

Thanks!

alextwoods commented 2 years ago

Thanks for reaching out on this - we do have plans to add support for BatchGetItems! As part of a new investment in this library we are planning to add support following the approach taken in PR #119 where we added support for batch writing items using a new BatchWrite class to avoid any backwards compatibility / breaking changes.

We are planning to add support for batch_get_item with a BatchRead class (capable of reading multiple items from multiple tables) which can be used by individual Record classes, giving an interface like:

class Foo
  include Aws::Record
  string_attr :name, hash_key: true
end

class Bar
  include Aws::Record
  string_attr :id, hash_key: true
  string_attr :rk, range_key: true
end

# produces a heterogeneous ItemCollection
items = Aws::Record::Batch.read do |db|
  db.find(Foo, name: 'n1')
  db.find(Foo, name: 'n2')
  db.find(Bar, id: 'id1', rk: 'rk1')
  db.find(Bar, id: 'id1', rk: 'rk2')
end

# returns a homogenous ItemCollection
foo_items = Foo.find_all(
  [
    {name: 'n1'},
    {name: 'n2'}
  ])

# an alternative syntax
foo_items = Foo.find_all do |db|
  db.find(name: 'n1') # the Foo class is implied here
  db.find(name: 'n2')
end

Would that work for your use case?

yyamanoi1222 commented 2 years ago

Thank you for your reply It looks like good!

One point of concern is that the existing ItemCollection class would have to be significantly modified. This is because BatchGetItem has no pagination and the batch_get_item response contains multiple table name keys. It seems simpler to return a simple array like transact_find than to abstract it with ItemCollection.

yyamanoi1222 commented 2 years ago

@alextwoods What do you think about the above?

And when will this function be implemented? I wonder if I should extend this gem to use it or wait for implementation. I can always help with implementation about this function. Sorry to bother you.

alextwoods commented 2 years ago

Sorry for the delay in response. Yes, the existing ItemCollection class wouldn't quite work for this case - and we would return an enumerable with a similar interface (supporting an each method) rather than modifying the ItemCollection class itself.

I believe we are currently targeting about mid Q4 of this year for the feature.

jterapin commented 1 year ago

Implemented the feature and published a new gem (v2.10.0).