search

slingdata-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
444 stars 34 forks source link

Feature/allow direct insert #411

Closed yokofly closed 4 weeks ago

yokofly commented 1 month ago

try fix #35 previous we work in #404 now here. usage: export SLING_ALLOW_DIRECT_INSERT=true current if we stop the insert to the target table, the rollback will do nothing, because the target table is directly inserted.

latest code 2 issues

  1. the column change, previous we have upsert to handle some column type incorrectly but still can be inserted from temp table to target, but now we do not have such a guard. so we need adjust column type directly from target if there is?(ideally we shall tell the user to create target table first by themselves?)

  2. the stream written count and the target table count. we shall find a more better way to comparison OR lower the err level to just logging instead of stopping the transfer. because in some cases we wrote ZERO count and the target table count is unrelated to this new insert.

flarco commented 4 weeks ago

Will work on branch v1.2.22, thanks