DistanceDevelopment / dssd

Distance Sampling Survey Design
1 stars 1 forks source link

Fix ATLAS issues from CRAN "Points of LinearRing do not form a closed linestring." #94

Closed LHMarshall closed 1 year ago

LHMarshall commented 1 year ago

See 'ATLAS' link in additional issues here - link now broken as dssd removed from CRAN.

This error appeared for both point and various line transect designs both in the generation of transects and also in the plotting of transects but did NOT always occur in each of these processes suggesting that the error was linked in part to the stochasticity present in dssd. Weird that it could sometimes generate transects and then not plot them though?

LHMarshall commented 1 year ago

Setup of the machine that the errors were found on:

Tests using R-devel on x86_64 Fedora 34 Linux with alternative BLAS/LAPACK implementations. Except as noted below, R was configured as per https://www.stats.ox.ac.uk/pub/bdr/Rconfig/r-devel-linux-x86_64-fedora-gcc

ATLAS: Serial ATLAS using the Fedora shared libraries. (Currently version 3.10.3.) libRblas.so was replaced by a symlink to /usr/lib64/atlas/libsatlas.so (so the LAPACK in the R sources was used). In earlier tests, similar results were obtained using ATLAS libs for the LAPACK and with a static BLAS (only).

MKL: Intel MKL '2022.0.2' used to build Rblas and also provides LAPACK (which is version 3.8.0). R was configured by (csh script)

setenv MKL_LIB_PATH /usr/local/MKL/mkl/lib/intel64 setenv LD_LIBRARY_PATH $MKL_LIB_PATH setenv MKL "-L$MKL_LIB_PATH -lmkl_gf_lp64 -lmkl_core -lmkl_sequential" ~/R/svn/R-devel/configure -C --with-blas="$MKL" --with-lapack --enable-lto=R

OpenBLAS:

Serial OpenBLAS using the Fedora shared libraries (currently version 0.3.19). libRblas.so was replaced by a symlink to /usr/lib64/libopenblas.so (part of openblas-devel, linked from openblas-serial).

LHMarshall commented 1 year ago

I downloaded the latest OpenBLAS release from here on to my windows machine unzipped it and replaced the Rblas.dll and Rlapack.dll files in the R installation with the libopenblas.dll in the bin folder of the downloaded .zip. R ran and it actually seemed to slow down benchmark functions! I couldn't replicate the errors found on CRAN.

I notice that sf is currently running into some CRAN check issues... see here although not with the ATLAS setup.

I am awaiting help from maths technical help to get the debian linux machine back on the network. Installation of BLAS and LAPACK looks simpler on that although R-devel not so easy.

LHMarshall commented 1 year ago

@lenthomas I missed it initially as there was no ATLAS issue reported in sf CRAN testing but turns out dssd is not the only package to have been struggling with this type of error. In fact the bug looks likely to stem from an issues in ATLAS! See this sf issue

The problem arises with rotation... which I do a lot of! I think usually 3 times per design when generating transects.

lenthomas commented 1 year ago

Thanks for the update. One option then is to leave off CRAN until the problem in ATLAS is fixed? Open to other suggestions!

LHMarshall commented 1 year ago

I've just added a comment on the sf issue to check if the issue has been reported to ATLAS. I also considered writing to CRAN and explaining that the errors are valid errors that something is genuinely wrong with upstream code and asking them to to put our packages back on CRAN.

lenthomas commented 1 year ago

BTW, did you see this as a possible work-around - something like st_make_valid(st_set_precision(x,1e8)).

LHMarshall commented 1 year ago

@lenthomas that is the plan. I would like to only mess with the precision on that particular system setup however. I also need to do that in quite a few places. The docker image will then let me test to make sure I've caught them all otherwise I have no way to check before sending it to CRAN.

LHMarshall commented 1 year ago

Note to self - when using the docker file the config.site file doesn't copy across from windows to unix properly... need to remove the line ending characters.

LHMarshall commented 1 year ago

Fixed