use TRUNCATE instead of DELETE FROM

coursera / dataduct

DataPipeline for humans.

Other

252 stars 82 forks source link

use TRUNCATE instead of DELETE FROM #109

Closed zhaojunz closed 9 years ago

zhaojunz commented 9 years ago

Redshift VACUUM steps seems to fail to claim space for DELETE FROM.

coveralls commented 9 years ago

Coverage increased (+0.05%) to 66.59% when pulling 9c7e6c1b5d5459b7d73911a2c50682b2c96a6be1 on zhaojunz:patch-1 into 269888a86143f176bb098cc86c6baaeaf0d50f5b on coursera:develop.

tpcstld commented 9 years ago

LGTM. @sb2nov?

sb2nov commented 9 years ago

LGTM if the transaction thing works

zhaojunz commented 9 years ago

It seems to work in transactions.

BTW, does this step in a transaction? if so, why? I feel like transaction would be a little excessive for this type of step as the table is reloaded anyway.

sb2nov commented 9 years ago

A transaction is important as if the step fails after the delete/truncate we don't want to leave an empty table there.