Open cboettig opened 3 years ago
Maybe with a callback function?
If we do it, it should be part of dbAppendTable()
. The way the default method is currently implemented (via sqlAppendTableTemplate()
) is abysmally slow for remote connections. The only portable solution seems to be to use dbQuoteLiteral()
to create a long SQL query, which is what sqlAppendTable()
does (or should do, r-dbi/DBI#362).
How long may these queries become? How to make the process RAM-friendly?
All of this is a lot of work and doesn't really fit the scope of DBI; doing this in the individual backends feels even worse. Maybe there's room for a package that specializes in writing/copying large datasets across DBI backends?
Is it possible to show a progress bar for long-running
dbWriteTable
commands?