search

qsbase / qs

Quick serialization of R objects
405 stars 19 forks source link

error when using data.table setnames(): ALTSTRING classes must provide a Set_elt method #9

Closed emilBeBri closed 5 years ago

emilBeBri commented 5 years ago

when saving a data.table with qsave, no matter which preset compression, and then using setnames() from data.table, the following error pops out:

Error in setnames(føl1, føl_reg$dan_s, føl_reg$dan) : ALTSTRING classes must provide a Set_elt method This error doesn't happen if setnames() is used directly on the data.table before it is saved and read with qsave/qread.

however by copying the DT with copy() then the error disappears. The error also disappears when creating a new variable, even by reference like so: DT[, var := 1L]

but it doesn't disapper just by making another reference to the same object with DT_copy <- DT , and it doesn't disappear by using setcolorder() before doing the setnames(). however, setcolorder works, it just doesn't change whatever needs to be changed in order for setnames() to work correctly.

I can't figure out which variables it is that creates this problem, since a simple 1 column data.table with a character vector does not create the error.

the dataset with the error attached, and can also be downloaded here: https://www.dropbox.com/s/16bhw6eplywcoej/egne_f%C3%B8lelser_datasheet.xlsx?dl=0

egne_følelser_datasheet.xlsx

traversc commented 5 years ago

I think I see what the problem is.

I don't think it's necessarily a bug with qs because data.table is calling C functions directly, but it will be fixed soon :) NVM it's a bug.

For now, when you read in the data, use the parameter use_alt_rep=F for now. 

library(qs)
library(readxl)
library(data.table)

setwd("~/Downloads")
x <- read_excel("egne_folelser_datasheet.xlsx")
x <- data.table(x)
qsave(x, file="temp.qs")
z <- qread("temp.qs", use_alt_rep = F)
setnames(z, 1, "hi") # no error hopefully
emilBeBri commented 5 years ago

allright, great! And thank you for a great package - better than feather AND fst, but with the flexibility with object-types from RDS - awesome!

emilBeBri commented 5 years ago

should I close the issue? I'm not that familiar with the procedures about these things.

traversc commented 5 years ago

I believe I've fixed the issue using altrep if you want to use it.

tbh, altrep doesn't really make too much difference, it's more for winning benchmarks :p

Feel free to re-open if there's still an issue.

And thanks for the compliments :)