qsbase / qs

Quick serialization of R objects
405 stars 19 forks source link

qs package looks not to support multibyte string path/filename at windows #17

Closed cuttlefish44 closed 5 years ago

cuttlefish44 commented 5 years ago

I'm Japanese and I use R with Rstudio at Windows 10. qs package always help me, thanks very much. But I found it have problems with filename/path of multibyte string. I would highly appreciate If multibyte path are supported.

qsave(iris, "あいりす.qs")
list.files()
# "縺ゅ>繧翫☆.qs"   # corrupted text

a <- qread("あいりす.qs") # But I can read it.

dir.create("中間データ")
qsave(iris, "中間データ/あいりす.qs") 
# Error: Failed to open file

Here is my sessionInfo().

sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale:
[1] LC_COLLATE=Japanese_Japan.932  LC_CTYPE=Japanese_Japan.932    LC_MONETARY=Japanese_Japan.932 LC_NUMERIC=C                  
[5] LC_TIME=Japanese_Japan.932    
traversc commented 5 years ago

This is a hard one, unfortunately I am not very familiar working with wide characters -- I'm having trouble replicating since I don't have a japanese windows locale. Does saveRDS(iris, "中間データ/あいりす.rds") work?

C++ standard doesn't directly understand wide characters as I understand, and I'm unsure of R's support on windows too.

In the mean time, could you define a new function like this:

qsave2 <- function(file, ...) {
  temp <- getwd()
  setwd(dirname(file))
  qsave(basename(file),...)
  setwd(temp)
}
cuttlefish44 commented 5 years ago

Thanks for your response. OK, I understood it is hard. I don't want you to take time to solve a language, not essential problem.

Does saveRDS(iris, "中間データ/あいりす.rds") work? Yes, it works.

I decided to make/use a custom function like you showd me kindly. Again, qs package is really helpful for me, THANKS FOR YOUR WORK !!

traversc commented 5 years ago

You're welcome! I'll leave this open if you don't mind to see if I can find a workaround eventually.

artemklevtsov commented 5 years ago

@cuttlefish44 can you provide path as bytes (with charToRaw)? I suppose the encoding on Windows is not UTF-8.

cuttlefish44 commented 5 years ago

@artemklevtsov; @traversc

Surprisingly, I can't reproduce the issue. I haven't changed my environment related R but Rstudio (I've done it update). This issue perhaps be caused by Rstudio. I just don't feel satisfied... but maybe my problem has been solved. If I encounter this problem again, I'll report it in more detail.

Thank you for your kindness.