Async Bulk Delete API

[aaguiarz]

Tuples can only be deleted individually.

If you want to delete a set of tuples, you need to first find them (by using the read API, or by construct them from data in your databases), and then call the write API with a set of tuples to delete.

We should provide a way to batch delete tuples, based on different conditions. Conditions should be consistent with the ones we have in the Read API.

This would be an async API, where you'd be able to query for status or get notified after it finishes.

Open questions:

• How should be the async behavior be modeled? Callback Webhook? Polling? Streaming callback?
• What should the API return? Number of tuples deleted + tuples with errors? All deleted tuples?
• Should we have a filter called 'Invalid Tuples' so I can delete all invalid tuples?

[ericmoraitis]

Adding my feedback after a discussion with Andres:

• Having the API return which tuples were deleted and which tuples returned errors would be beneficial. It would provide the ability to do something about the tuples that threw errors, while also confirming which tuples were successfully deleted.
• One primary use case we have is: user A might need to delete X number of objects with relation Y.
  • This is similar to the current write endpoint, but there is a hard limit of 40 tuples with that endpoint.
  • We have cases where a given user might want to delete > 40 objects at a time, which would require sending >40 tuples to the endpoint at once (but not necessarily delete ALL items for the given relation).
  • As a result, we are having to loop through this endpoint in batches of 40, which may fail in one of the loops, resulting in some batches of tuples being deleted, and others not. This also presents problems in the case some tuple relationships do not exist to begin with, and the entire batch fails, even if some relationships could have been deleted.
  • Ideally, the endpoint would support deleting >40 tuples at once, and delete the tuples that can be deleted. It would return success/failure for each tuple, and error messages for each of the failed tuples.

Hope this helps!

[rhamzeh]

@ericmoraitis have you looked at our SDKs by any chance? Curious to know your thoughts about the semantics there.

All 5 of them support what we call non-transactional writes if you pass in the proper options. See here

We're trying to understand if: 1) the semantics exposed by the SDK are good enough as an interface or if we should iterate on them (any future API would probably use the same semantics) 2) Whether there is a need for this to be handled by the API instead of the SDK

If you are using one of the supported languages (JS, Go, .NET, Python, Java), try them out and let us know!

[aaguiarz]

A similar idea would be to provide a 'Bulk Transform API', were you can specify a query and a transformation to apply (e.g. for each tuple you find, add a transformed one).

This can be useful for model migrations where you need to rename a relation/tuple. In those scenario you'd first want to create tuples for the new relation/type, then deploy the new version of your app that uses and point to the new model.

You'd delete the tuples after the new version is deployed.