read.las fails when path contains special characters on windows with v1.3.2

[jfbourdon]

Linked to issue #38, it seems that, at least on Windows, the path given to read.las cannot contain any accent. rlas 1.2.7 and 1.2.8 work as expected, but 1.3.2 doesn't.

rlas::read.las("E:/é/clip_LiDAR.laz")
#> Error in C_reader(ifiles, ofile, select, filter, filter_wkt) : 
#>   LASlib internal error. See message above.

[Jean-Romain]

Paths with diacritic symbols should be punish by penalties. Sadly I do not write the laws :wink:

Let make some tests. Copy paste the following. I have 0 error but you will probably have 2 errors for test2 and test3. Is that correct?

test1 <- system.file("extdata", "example.laz", package="rlas")
test2 <- paste0(tempdir(), "/abcdé/test.las")
test3 <- paste0(tempdir(), "/abcd/testé.las")

dir2  <- dirname(test2)
dir3  <- dirname(test3)
dir.create(dir2)
dir.create(dir3)
file.copy(test1, test2)
file.copy(test1, test3)

x <- rlas::read.las(test1)                 
x <- rlas::read.las(test2)  
x <- rlas::read.las(test3)  

[bi0m3trics]

As a predominately windows user and the owner of a surname with both an accent and a space… I gave up on this a long time ago... ;)

[Jean-Romain]

@bi0m3trics do you encounter issues on Windows like @jfbourdon?

[jfbourdon]

I agree that diacritic symbols should be avoided in pathnames... but sometimes they are there!

test1 -> pass test2 -> fail test3 -> fail

[Jean-Romain]

Which tests have you made? Different R versions? Different rlas versions? Please show me the different combinations you tested? The source code has been not changed so if it comes from the code it is likely to be deeply hidden in laslib. So I prefer to be sure it is not related to the recent release of R 3.6.0 and encoding modifications before to investigate further.

[jfbourdon]

R 3.4.3 + rlas 1.2.8 => pass R 3.4.3 + rlas 1.3.2 => fail R 3.5.1 + rlas 1.2.7 => pass R 3.5.1 + rlas 1.3.2 => fail R 3.6.0 + rlas 1.3.2 => fail

I would like to compile an older version of rlas on R 3.6.0 but Rtools gives me trouble; I highly suspect a problem caused by the way our IT department setup Windows and securities. I'll continue to try finding a way.

[bi0m3trics]

@Jean-Romain I would not use special characters… but since you asked… I copied "MixedConifer.laz" to a folder called “Sánchez Meador” and ran the following:

las<-readLAS("C:/Dropbox/Temp/Sánchez Meador/MixedConifer.laz") and received the following error:

Error: LASlib internal error. See message above.
Error in C_reader(ifiles, ofile, select, filter, filter_wkt) : 
  LASlib internal error. See message above.

Running:

R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
And rlas_1.3.2  lidR_2.1.0

I then removed the space but left the special character, and it gave me the same error, and then I removed the special character and put the space back and it ran fine (which I expected).

[Jean-Romain]

I reproduced on my virtual machine with the example I wrote above. Bug confirmed.

[jfbourdon]

I finally managed to get Rtools working and made more tests:

R 3.4.3 + rlas 1.3.2 (from source) => pass R 3.4.4 + rlas 1.3.2 (from source) => pass

R 3.5.0 + rlas 1.2.7 (from source) => fail R 3.5.0 + rlas 1.2.9 (from source) => fail R 3.5.0 + rlas 1.3.0 (from source) => fail R 3.5.0 + rlas 1.3.1 (from source) => fail R 3.5.0 + rlas 1.3.2 (from source) => fail

R 3.5.1 + rlas 1.3.2 (from source) => fail R 3.5.2 + rlas 1.3.2 (from source) => fail R 3.5.3 + rlas 1.3.2 (from source) => fail

R 3.6.0 + rlas 1.2.7 (from source) => fail R 3.6.0 + rlas 1.3.2 (from source) => fail

So it may be more related to R than to rlas. I found this issue with package readr that may have the same cause. With R 3.4.4 Encoding(normalizePath("E:/é/clip_LiDAR.laz")) gives "unknown" but with R 3.5.0 it gives "UTF-8".

[Jean-Romain]

Thank you @jfbourdon . I actually spotted the issue and its location but I was not able to reproduce it in simple cases. Without your link I would not have found that normalizePath was the source of the issue by modifying the native encoding of the string. I fixed the issue by re-encoding to native encoding. Should work.

The point is that this problem actually happen from R 3.5.0 so almost a year and nobody reported it.