search

citusdata / cstore_fdw

Columnar storage extension for Postgres built as a foreign data wrapper. Check out https://github.com/citusdata/citus for a modernized columnar storage implementation built as a table access method.
Apache License 2.0
1.76k stars 171 forks source link

[question] Atomic transactions #179

Open kostiantyn-nemchenko opened 6 years ago

kostiantyn-nemchenko commented 6 years ago

Hi,

I have noticed that once the foreign table has been populated with data within an open transaction the changes will not be rolled back if the transaction is aborted. Do you have plans to change this behavior in future?

Thank you

mtuncer commented 6 years ago

Hi @kostiantyn-nemchenko

Thanks for asking, I have been thinking about this for some time. We have an implementation on develop branch with limited support for transactions. Full (or better) support for transactions is the major step before the next release.

kostiantyn-nemchenko commented 6 years ago

@mtuncer , thanks for your response. Good to know that this is on your radar. Will wait for the next release.

PerhunAndrey commented 4 years ago

Hello, Regarding this feature. What will happen if write operation will be aborted in the midst and not all rows will be written in a stripe/block? Is there any way to guarantee the idempotence of a write operation, taking into account that there is no support of DELETE?

pykello commented 4 years ago

@PerhunAndrey when doing INSERT ... SELECT or COPY into a cstore table, either all rows get written to the cstore table or none. That means if you cancel the write operation or server crashes while writing, none of the rows written by the failed write operation will be visible.

If you want more details, what we do is:

  1. Add rows to the data file
  2. Atomic update of metadata file (using rename system call) to make the new stripes visible.

Rows won't be visible until step 2 is completed.

If step 1 is done or partially done and step 2 didn't happen because of a crash or cancellation, future writes will overwrite the invisible rows. So we won't have any wasted space in long term.