ElliottKasoar / abcd

0 stars 0 forks source link

Investigate rollback #26

Open ElliottKasoar opened 7 months ago

ElliottKasoar commented 7 months ago

If operations (insertion, deletion) fail, we want to handle this gracefully. Ideally we would be able to rollback changes, but this does not appear to be possible directly through OpenSearch.

It had been suggested that bulk operations may enable this, which is partially the case. For example, it appears that failure mid-insertion of a chunk will mean none of that chunk is inserted, but in cases where multiple chunks are required, some chunks may still be uploaded.

This is still considerably neater, as we can add options to set the request_timeout and size, and potentially tell users explicitly how much data was uploaded.