Transaction support

[sidtaduri]

Heya @jlafon,

Any plans to support transactions in future releases? https://aws.amazon.com/blogs/aws/dynamodb-transaction-library/

[jlafon]

Hi @sidtaduri, I had read through the code in that library when it was released to see if PynamoDB could provide a similar feature. It's not a trivial thing to implement, and I don't have quite enough time to dedicate to it (I maintain PynamoDB in my spare time). However, pull requests are always welcome :)

[jmphilli]

@sidtaduri do you still feel you need this functionality?

[sidtaduri]

@jmphilli - not at the moment.

[wobeng]

@jmphilli I do, do you have it?

[jhesed]

Hi @jmphilli, Is there any plan on implementing transactions in the near future?

[vinodmehra]

https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transactions/?sc_channel=sm&sc_publisher=TWITTER&sc_country=Global&sc_geo=GLOBAL&sc_outcome=awareness&trk=AWS_reInvent_2018_launch_DynamoDB_Transactions_TWITTER&sc_content=AWS_reInvent_2018_launch_DynamoDB_Transactions&sc_category=Amazon+DynamoDB&linkId=60250973

[jhesed]

wow that's a gem @vinodmehra. Thanks!

[batmop]

Would love this implemented

[disrael]

This is a must have. Please assign and give time frame or give us pointers on how we would pull request it in.

[Surgo]

I think it have following tasks

• [ ] Share connection between tables (maybe backward imcompatible)
  • TransactGetItems and TransactWriteItems can updating multiple tables at once
  • Like django's global db connection?
• [ ] Add TransactGetItems TransactWriteItems
  • Split current update_item like methods into get_update_item_params (using in TransactWriteItems) and update_item

So pynamodb.connection.base.Connection API become

class Connection():
   def get_update_item_operation_kwargs(self, *args, **kwargs):
      // Create operation_kwargs to update
      return operation_kwargs

   def update_item(self, *args, **kwargs):
     operation_kwargs = self.get_update_item_operation_kwargs(*args, **kwargs)
        try:
            return self.dispatch(UPDATE_ITEM, operation_kwargs)
        except BOTOCORE_EXCEPTIONS as e:
            raise UpdateError("Failed to update item: {0}".format(e), e)

   def transact_write(self, table_operations):
      transaction_items = []
      for table, kwargs in table_operations.items():
         if operation['...']  == UPDATE_ITEM':
            transaction_items.append(table, self.get_update_item_operation_kwargs(*kwargs))
      return self.dispatch(TRANSACTION_WRITE_ITEM, {'TransanctionItems': transaction_items})

Usage

from pynamodb import connection

connection.transact_write([
    Table.update_item(,,,),
])

[jhesed]

Any update regarding this? Thanks!

[jaredtkatz]

Yeah this would be super useful

[emise]

yes please!

[toshke]

+1 - anyone working on this, is it on the roadmap? I'm keen to help if required.

[garrettheel]

I'd definitely love to add support for this to PynamoDB. I'm not aware of anyone working on it thus far, but I'm happy to work with anyone that wants to take a crack at this

[hallie]

For those of you following along, I'm interested in building out support for this feature, but would love some feedback on a couple of different ways to interface with it:

Approach 1

Get

appointment, user = pynamodb.transaction.get_transaction(
    transact_items=[
        Appointment.get_transact(hash_key=5, range_key=123456),
        User.get_transact(hash_key=3, range_key=None),
    ],
    client_request_token="foo" ,
)

Write

new_appointment = Appointment(current_state="QUEUED", is_scheduled=True)
response = pynamodb.transaction.write_transaction(
    transact_items=[
        new_appointment.save_transact(condition=(Appointment.id.does_not_exist())),
        Appointment.update_transact(
            hash_key=5,
            range_key=123456,
            condition=(Appointment.current_state != "ENDED"),
            action=(Appointment.current_state.set("ENDED"))
        ),
        User.condition_transact(hash_key=5, condition=(~User.is_deactivated)),
        User.delete_transact(User, hash_key=6, condition=(User.phone_number == dupe_number)),
    ],
    client_request_token="foo",
)

Approach 2

Get

with pynamodb.transaction.get_transaction(client_request_token="foo") as t:
    t.add_get(Appointment, hash_key=5, range_key=123456)
    t.add_get(User, hash_key=5, range_key=None)
appointment, user = t.results()

Write

with pynamodb.transaction.write_transaction(client_request_token="foo") as t:
    t.add_save(
        Appointment,
        id=id.generate(),
        created_at_ms=utc_now_ms(),
        current_state="QUEUED",
        is_scheduled=True
    )
    t.add_update(
        Appointment,
        hash_key=5,
        range_key=123456,
        condition=(Appointment.current_state != "ENDED"),
        action=(Appointment.current_state.set("ENDED"))
    )
    t.add_condition(User, hash_key=5, condition=(~User.is_deactivated))
    t.add_delete(User, hash_key=6, condition=(User.phone_number == dupe_number))
response = t.result()

Approach 3

Get

transaction = pynamodb.transaction.get_transaction(client_request_token="foo")
Appointment.get(hash_key=5, range_key=123456, with=transaction)
User.get(hash_key=3, range_key=None, with=transaction)
appointment, user = transaction.result()

Write

transaction = pynamodb.transaction.write_transaction(client_request_token="foo")
new_appointment = Appointment(current_state=”QUEUED”, is_scheduled=True)
new_appointment.save(with=transaction, condition=(Appointment.id.does_not_exist()))
Appointment.update(
    with=transaction,
    hash_key=5, range_key=123456,
    condition=(Appointment.current_state != "ENDED"),
    action=(Appointment.current_state.set("ENDED"))
)
User.condition(hash_key=5, condition=(~User.is_deactivated), with=transaction)
User.delete(User, hash_key=6, condition=(User.phone_number == dupe_number), with=transaction)
response = transaction.result()

[ikonst]

All Get variants are difficult to type check.

Here are some more ideas to bounce off you:

with start_get_transaction(client_request_token="foo") as t:
    appointment = Appointment.get(5, 123456, transaction=t)
    user = User.get(3, transaction=t)
    # the objects are in special limbo state: any property access will raise a runtime error

# user.phone_number can be accessed now

I'm not sure I like the limbo state and errors that can only be detected at runtime, so here's another idea:

with start_get_transaction(client_request_token="foo") as t:
    t.add_get(Appointment, hash_key=5, range_key=123456)
    t.add_get(User, hash_key=5, range_key=None)

user = t.results.get(User, 5)  # type stub can define 'get(Type[T], ...) -> T`
# t.results.get(User, 6) would've raised an error since there's no such result

appointment = t.results.get(Appointment, 5, 123456)

[wobeng]

any update on this feature?

[ikonst]

@wobeng It's being actively developed and will likely land in the master branch (i.e. the 4.0 beta) next month. Watch #618.

[jaredtkatz]

Nice!

On Fri, Jun 28, 2019 at 12:22 PM Ilya Konstantinov notifications@github.com wrote:

@wobeng https://github.com/wobeng It's being actively developed and will likely land in the master branch (i.e. the 4.0 beta) next month. Watch

618 https://github.com/pynamodb/PynamoDB/pull/618.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pynamodb/PynamoDB/issues/89?email_source=notifications&email_token=AB7I2YT4S6WBBVA7YK3ZJS3P4Y3EZA5CNFSM4BTT5IYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2Q6UY#issuecomment-506793811, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7I2YSY2HCZ4K5XUQAOSSTP4Y3EZANCNFSM4BTT5IYA .

[wobeng]

@ikonst thank you...for all you do