ankane / dbx

A fast, easy-to-use database library for R
Other
187 stars 15 forks source link

Support for tibble data type #11

Closed mowreyw closed 5 years ago

mowreyw commented 5 years ago

Thanks for your work in putting together this great package! It would be great if these functions also supported the tibble datatype, in addition to data.frame. I've been caught out a few times by functions that returned a tibble (when I was expecting a data.frame), and when I attempt a dbxUpdate, dbxUpsert, or dbxInsert operation I get the following error:

Error: Lists must contain raw vectors or NULL

The traceback is typically something like the following:

11: stop("Lists must contain raw vectors or NULL", call. = FALSE)
10: FUN(X[[i]], ...)
9: vapply(x, function(x) {
       if (is.null(x)) {
           "NULL"
       }
       else if (is.raw(x)) {
           paste0("X'", paste(format(x), collapse = ""), "'")
       }
       else {
           stop("Lists must contain raw vectors or NULL", call. = FALSE)
       }
   }, character(1))
8: DBI::dbQuoteLiteral(conn, col)
7: DBI::dbQuoteLiteral(conn, col)
6: quoteRecords(conn, records)
5: valuesClause(conn, records)
4: insertClause(conn, table, batch)
3: f(records)
2: inBatches(records, batch_size, function(batch) {
       if (isMySQL(conn)) {
           sql <- insertClause(conn, table, batch)
           if (skip_existing) {
               set_sql <- upsertSetClause(quoted_where_cols)
           }
           else {
               set_sql <- upsertSetClause(quoted_update_cols)
           }
           sql <- paste(sql, "ON DUPLICATE KEY UPDATE", set_sql)
           selectOrExecute(conn, sql, batch, returning = returning)
       }
       else {
           conflict_target <- colsClause(quoted_where_cols)
           sql <- insertClause(conn, table, batch)
           sql <- paste0(sql, " ON CONFLICT (", conflict_target, 
               ") DO")
           if (skip_existing) {
               sql <- paste(sql, "NOTHING")
           }
           else {
               set_sql <- upsertSetClausePostgres(quoted_update_cols)
               sql <- paste(sql, "UPDATE SET", set_sql)
           }
           selectOrExecute(conn, sql, batch, returning = returning)
       }
   })
1: dbx::dbxUpsert(conn = con, table = "xxx", records = some_tibble, 
       where_cols = c("yyy"))

This can be observed by creating any arbitrary tibble and attempting any of the above dbx operations. Coercing to data.frame resolves the Lists must contain raw vectors or NULL error in all cases that I've encountered. Coercing to data.frame is easy enough, but it would be great if the package supported the tibble type.

ankane commented 5 years ago

Hey @mowreyw, thanks for the suggestion 👍 Added support on master.