barracuda156
commented
6 months ago Closed barracuda156 closed 5 months ago
barracuda156
commented
6 months ago barracuda156
commented
6 months ago Hmm, it is weird. CRAN version 0.3.2 builds fine, but 0.3.2.1 from here 7853a3ef17eee41777fa3f8f2548981bfb8970e1 fails already. However there seems to be no 0.3.2 here at all.
barracuda156
commented
6 months ago Looks like the last non-CRAN version to build is 0.3.1.2, i.e. ad154c2dd1b724a7306b3e2d601040afefd6836a commit:
---> Building R-dqrng
xinstall: mkdir /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build
Executing: cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng-ad154c2dd1b724a7306b3e2d601040afefd6836a" && /opt/local/bin/R CMD INSTALL . --library=/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build --install-tests
* installing *source* package ‘dqrng’ ...
** using staged installation
** libs
using C++ compiler: ‘g++-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0’
Warning in system2("xcrun", "--show-sdk-path", TRUE, TRUE) :
running command ''xcrun' --show-sdk-path 2>&1' had status 64
using SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c RcppExports.cpp -o RcppExports.o
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c dqrng.cpp -o dqrng.o
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c generateSeedVectors.cpp -o generateSeedVectors.o
/opt/local/bin/g++-mp-13 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/opt/local/Library/Frameworks/R.framework/Resources/lib -Wl,-headerpad_max_install_names -L/opt/local/lib -lMacportsLegacySupport -Wl,-rpath,/opt/local/lib/libgcc -arch ppc -o dqrng.so RcppExports.o dqrng.o generateSeedVectors.o -F/opt/local/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build/00LOCK-dqrng-ad154c2dd1b724a7306b3e2d601040afefd6836a/00new/dqrng/libs
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (dqrng)0.3.1.3, i.e. commit abdb3ebba26dc78f0f0502c6c10ab894ae46a30e – is already broken:
---> Building R-dqrng
xinstall: mkdir /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build
Executing: cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng-abdb3ebba26dc78f0f0502c6c10ab894ae46a30e" && /opt/local/bin/R CMD INSTALL . --library=/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build --install-tests
* installing *source* package ‘dqrng’ ...
** using staged installation
** libs
using C++ compiler: ‘g++-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0’
Warning in system2("xcrun", "--show-sdk-path", TRUE, TRUE) :
running command ''xcrun' --show-sdk-path 2>&1' had status 64
using SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c RcppExports.cpp -o RcppExports.o
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c dqrng.cpp -o dqrng.o
In file included from ../inst/include/dqrng_generator.h:25,
from dqrng.cpp:21:
../inst/include/pcg_random.hpp: In instantiation of 'void pcg_detail::specific_stream<itype>::set_stream(itype) [with itype = pcg_extras::uint_x4<unsigned int, long long unsigned int>]':
../inst/include/pcg_random.hpp:610:26: required from 'std::basic_istream<_CharT, _Traits>& pcg_detail::operator>>(std::basic_istream<_CharT, _Traits>&, engine<xtype, itype, output_mixin, output_previous, stream_mixin, multiplier_mixin>&) [with CharT = char; Traits = std::char_traits<char>; xtype = long long unsigned int; itype = pcg_extras::uint_x4<unsigned int, long long unsigned int>; output_mixin = xsl_rr_mixin<long long unsigned int, pcg_extras::uint_x4<unsigned int, long long unsigned int> >; bool output_previous = false; stream_mixin = specific_stream<pcg_extras::uint_x4<unsigned int, long long unsigned int> >; multiplier_mixin = default_multiplier<pcg_extras::uint_x4<unsigned int, long long unsigned int> >]'
../inst/include/dqrng_generator.h:56:55: required from 'void dqrng::random_64bit_wrapper<RNG>::input(std::istream&) [with RNG = pcg_detail::engine<long long unsigned int, pcg_extras::uint_x4<unsigned int, long long unsigned int>, pcg_detail::xsl_rr_mixin<long long unsigned int, pcg_extras::uint_x4<unsigned int, long long unsigned int> >, false, pcg_detail::specific_stream<pcg_extras::uint_x4<unsigned int, long long unsigned int> >, pcg_detail::default_multiplier<pcg_extras::uint_x4<unsigned int, long long unsigned int> > >; std::istream = std::basic_istream<char>]'
../inst/include/dqrng_generator.h:56:16: required from here
../inst/include/pcg_random.hpp:322:37: error: no match for 'operator|' (operand types are 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' and 'int')
322 | inc_ = (specific_seq << 1) | 1;
| ~~~~~~~~~~~~~~~~~~~~^~~
../inst/include/pcg_random.hpp:322:37: note: candidate: 'operator|(int, int)' (built-in)
../inst/include/pcg_random.hpp:322:37: note: no known conversion for argument 1 from 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' to 'int'
../inst/include/pcg_random.hpp:322:37: note: candidate: 'operator|(long unsigned int, int)' (built-in)
../inst/include/pcg_random.hpp:322:37: note: no known conversion for argument 1 from 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' to 'long unsigned int'
../inst/include/pcg_random.hpp:322:37: note: candidate: 'operator|(unsigned int, int)' (built-in)
../inst/include/pcg_random.hpp:322:37: note: no known conversion for argument 1 from 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' to 'unsigned int'
../inst/include/pcg_random.hpp:322:37: note: candidate: 'operator|(long long unsigned int, int)' (built-in)
../inst/include/pcg_random.hpp:322:37: note: no known conversion for argument 1 from 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' to 'long long unsigned int'
In file included from ../inst/include/pcg_extras.hpp:84,
from ../inst/include/pcg_random.hpp:114:
../inst/include/pcg_uint128.hpp:664:22: note: candidate: 'template<class UInt, class UIntX2> pcg_extras::uint_x4<U, V> pcg_extras::operator|(const uint_x4<U, V>&, const uint_x4<U, V>&)'
664 | uint_x4<UInt,UIntX2> operator|(const uint_x4<UInt,UIntX2>& a,
| ^~~~~~~~
../inst/include/pcg_uint128.hpp:664:22: note: template argument deduction/substitution failed:
../inst/include/pcg_random.hpp:322:37: note: mismatched types 'const pcg_extras::uint_x4<U, V>' and 'int'
322 | inc_ = (specific_seq << 1) | 1;
| ~~~~~~~~~~~~~~~~~~~~^~~
In file included from /opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include/Rcpp/sugar/logical/logical.h:30,
from /opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include/Rcpp/sugar/sugar_forward.h:29,
from /opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include/RcppCommon.h:184,
from /opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include/Rcpp.h:27,
from dqrng.cpp:20:
/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include/Rcpp/sugar/logical/or.h:315:1: note: candidate: 'template<bool LHS_NA, class LHS_T, bool RHS_NA, class RHS_T> Rcpp::sugar::Or_LogicalExpression_LogicalExpression<LHS_NA, LHS_T, RHS_NA, RHS_T> operator|(const Rcpp::VectorBase<10, LHS_NA, LHS_T>&, const Rcpp::VectorBase<10, RHS_NA, RHS_T>&)'
315 | operator|(
| ^~~~~~~~
/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include/Rcpp/sugar/logical/or.h:315:1: note: template argument deduction/substitution failed:
../inst/include/pcg_random.hpp:322:37: note: 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' is not derived from 'const Rcpp::VectorBase<10, LHS_NA, LHS_T>'
322 | inc_ = (specific_seq << 1) | 1;
| ~~~~~~~~~~~~~~~~~~~~^~~
make: *** [dqrng.o] Error 1
ERROR: compilation failed for package ‘dqrng’
* removing ‘/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build/dqrng’
Command failed: cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng-abdb3ebba26dc78f0f0502c6c10ab894ae46a30e" && /opt/local/bin/R CMD INSTALL . --library=/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build --install-tests
Exit code: 1
Error: Failed to build R-dqrng: command execution failedHowever whatever CRAN has as 0.3.2 builds with no issues.
@rstub @daqana Could someone please help me here?
rstub
commented
6 months ago v0.3.2 is https://github.com/daqana/dqrng/commit/bfcf168907bc3001266243c67dad6c1096090cb6. Due to an issue with Rcpp, v0.3.2 had to be directly based on v0.3.1 instead of the development versions v0.3.1.x.
Can you tell me more about the environment, especiall0.3.1.3y the compiler, you are using? The code the build of 0.3.1.3 is complaining about has been in the package since the very beginning. Of course, it might be that it had never been compiled. My guess: Your compiler does not support 128 bit integers so that the implementation in in pcg_uint128.hpp is used
barracuda156
commented
6 months ago v0.3.2 is bfcf168. Due to an issue with Rcpp, v0.3.2 had to be directly based on v0.3.1 instead of the development versions v0.3.1.x.
Can you tell me more about the environment, especiall0.3.1.3y the compiler, you are using? The code the build of 0.3.1.3 is complaining about has been in the package since the very beginning. Of course, it might be that it had never been compiled. My guess: Your compiler does not support 128 bit integers so that the implementation in in
pcg_uint128.hppis used
@rstub Thank you for responding!
I am using gcc 13.2.0, but this is a 32-bit (Big-endian) arch, and apparently uint128 is not supported for 32-bit: https://stackoverflow.com/questions/16088282/is-there-a-128-bit-integer-in-gcc
I could raise an issue with GCC upstream, if some further clarification is needed.
But it looks like 32-bit arch may need to fall back to 64-bit integers. Would it be possible to have such a codepath for better compatibility?
P. S. For the context, I am maintaining it in MacPorts, and it would be nice not to be forced to peg a package to an older version. We support ppc and i386, and there are people using arm and mips too, I believe.
rstub
commented
6 months ago Thanks for confirming my guess. I probably should update PCG to get https://github.com/imneme/pcg-cpp/commit/66478a9e7e8a81085ba71f33df51fdc0a1cd11c0. Then I can test without compiler internal 128bit integers.
Due to some UBSAN failures (c.f. #89) I will have to build a version 0.4.1 soon. I will try to provide a fix for this in there as well.
BTW, how is the performance of dqrng on 32bit architectures?
barracuda156
commented
6 months ago @rstub Tests pass (for CRAN-sourced 0.3.2):
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)
> library(testthat)
> library(dqrng)
>
> test_check("dqrng")
[ FAIL 0 | WARN 0 | SKIP 5 | PASS 136 ]
══ Skipped tests (5) ═══════════════════════════════════════════════════════════
• No long-vector support (5): 'test-sample.R:74:5', 'test-sample.R:86:5',
'test-sample.R:98:5', 'test-sample.R:110:5', 'test-sample.R:139:5'
[ FAIL 0 | WARN 0 | SKIP 5 | PASS 136 ]
>
> proc.time()
user system elapsed
46.424 6.768 54.293 Overall output of checks:
---> Testing R-dqrng
Executing: cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng" && /opt/local/bin/R CMD check ./dqrng_0.3.2.tar.gz --no-manual --no-build-vignettes
* using log directory ‘/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng/dqrng.Rcheck’
* using R version 4.4.0 (2024-04-24)
* using platform: powerpc-apple-darwin10.0.0d2 (32-bit)
* R was compiled by
gcc-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0
GNU Fortran (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0
* running under: OS X Snow Leopard 10.6
* using session charset: UTF-8
* using options ‘--no-manual --no-build-vignettes’
* checking for file ‘dqrng/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘dqrng’ version ‘0.3.2’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘dqrng’ can be installed ... OK
* used C++ compiler: ‘g++-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0’
* used SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’
* checking installed package size ... OK
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking whether startup messages can be suppressed ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in shell scripts ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking compiled code ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
Running ‘testthat.R’
OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking running R code from vignettes ...
‘cpp-api.Rmd’ using ‘UTF-8’... OK
‘dqrng.Rmd’ using ‘UTF-8’... OK
‘parallel.Rmd’ using ‘UTF-8’... OK
‘sample.Rmd’ using ‘UTF-8’... OK
OK
* checking re-building of vignette outputs ... SKIPPED
* DONE
Status: OKCan you try the version from #90? https://github.com/daqana/dqrng/pull/90/commits/fd688fb4c28ac8d36b9b7cf5ce689648eb5f66d7 should fix this issue.
barracuda156
commented
6 months ago @rstub The build from fd688fb4c28ac8d36b9b7cf5ce689648eb5f66d7 works, though with some warnings:
---> Building R-dqrng
xinstall: mkdir /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build
Executing: cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng-fd688fb4c28ac8d36b9b7cf5ce689648eb5f66d7" && /opt/local/bin/R CMD INSTALL . --library=/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build --install-tests
* installing *source* package ‘dqrng’ ...
** using staged installation
** libs
using C++ compiler: ‘g++-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0’
Warning in system2("xcrun", "--show-sdk-path", TRUE, TRUE) :
running command ''xcrun' --show-sdk-path 2>&1' had status 64
using SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c RcppExports.cpp -o RcppExports.o
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c dqrng.cpp -o dqrng.o
In file included from ../inst/include/dqrng_generator.h:25,
from dqrng.cpp:21:
In member function 'void dqrng::xoshiro<N>::do_jump(std::array<long long unsigned int, N>) [with long unsigned int N = 2]',
inlined from 'void dqrng::xoshiro<N>::jump() [with long unsigned int N = 2]' at ../inst/include/xoshiro.h:75:12:
../inst/include/xoshiro.h:112:21: warning: iteration 2 invokes undefined behavior [-Waggressive-loop-optimizations]
112 | if (JUMP[i] & UINT64_C(1) << b) {
../inst/include/xoshiro.h:110:31: note: within this loop
110 | for(unsigned int i = 0; i < sizeof JUMP / sizeof JUMP.begin(); i++)
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In member function 'void dqrng::xoshiro<N>::do_jump(std::array<long long unsigned int, N>) [with long unsigned int N = 4]',
inlined from 'void dqrng::xoshiro<N>::long_jump() [with long unsigned int N = 4]' at ../inst/include/xoshiro.h:81:12:
../inst/include/xoshiro.h:131:21: warning: iteration 4 invokes undefined behavior [-Waggressive-loop-optimizations]
131 | if (JUMP[i] & UINT64_C(1) << b) {
../inst/include/xoshiro.h:129:31: note: within this loop
129 | for(unsigned int i = 0; i < sizeof JUMP / sizeof JUMP.begin(); i++)
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -DSTRICT_R_HEADERS -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/BH/include' -I'/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/sitmo/include' -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c generateSeedVectors.cpp -o generateSeedVectors.o
/opt/local/bin/g++-mp-13 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/opt/local/Library/Frameworks/R.framework/Resources/lib -Wl,-headerpad_max_install_names -L/opt/local/lib -lMacportsLegacySupport -Wl,-rpath,/opt/local/lib/libgcc -arch ppc -o dqrng.so RcppExports.o dqrng.o generateSeedVectors.o -F/opt/local/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/build/00LOCK-dqrng-fd688fb4c28ac8d36b9b7cf5ce689648eb5f66d7/00new/dqrng/libs
** RAnd this:
* checking compiled code ... WARNING
File ‘dqrng/libs/dqrng.so’:
Found ‘_abort’, possibly from ‘abort’ (C)
Object: ‘dqrng.o’
Compiled code should not call entry points which might terminate R nor
write to stdout/stderr instead of to the console, nor use Fortran I/O
nor system RNGs nor [v]sprintf.There is a failure in tests now, though it may not be in fact related to dqrng at all:
* checking tests ...
Running ‘testthat.R’/opt/local/Library/Frameworks/R.framework/Resources/bin/BATCH: line 60: 15027 Bus error ${R_HOME}/bin/R -f ${in} ${opts} ${R_BATCH_OPTIONS} > ${out} 2>&1
ERROR
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
26: tryCatch(withCallingHandlers({ eval(code, test_env) if (!handled && !is.null(test)) { skip_empty() }}, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, message = handle_message, error = handle_error), error = handle_fatal, skip = function(e) { })
27: test_code(test = NULL, code = exprs, env = env, default_reporter = StopReporter$new())
28: source_file(path, env = env(env), desc = desc, error_call = error_call)
29: FUN(X[[i]], ...)
30: lapply(test_paths, test_one_file, env = env, desc = desc, error_call = error_call)
31: doTryCatch(return(expr), name, parentenv, handler)
32: tryCatchOne(expr, names, parentenv, handlers[[1L]])
33: tryCatchList(expr, classes, parentenv, handlers)
34: tryCatch(code, testthat_abort_reporter = function(cnd) { cat(conditionMessage(cnd), "\n") NULL})
35: with_reporter(reporters$multi, lapply(test_paths, test_one_file, env = env, desc = desc, error_call = error_call))
36: test_files_serial(test_dir = test_dir, test_package = test_package, test_paths = test_paths, load_helpers = load_helpers, reporter = reporter, env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, desc = desc, load_package = load_package, error_call = error_call)
37: test_files(test_dir = path, test_paths = test_paths, test_package = package, reporter = reporter, load_helpers = load_helpers, env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, load_package = load_package, parallel = parallel)
38: test_dir("testthat", package = package, reporter = reporter, ..., load_package = "installed")
39: test_check("dqrng")
An irrecoverable exception occurred. R is aborting now ...
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... WARNING
Directory 'inst/doc' does not exist.
Package vignettes without corresponding single PDF/HTML:
‘cpp-api.Rmd’
‘dqrng.Rmd’
‘parallel.Rmd’
‘sample.Rmd’
* checking running R code from vignettes ...
‘cpp-api.Rmd’ using ‘UTF-8’... OK
‘dqrng.Rmd’ using ‘UTF-8’... OK
‘parallel.Rmd’ using ‘UTF-8’... OK
‘sample.Rmd’ using ‘UTF-8’... OK
OK
* checking re-building of vignette outputs ... SKIPPED
* DONE
Status: 1 ERROR, 3 WARNINGsIt looks very similar to this bug in R: https://trac.macports.org/ticket/67059
I have no idea what is that and why it causes Bus error, but it seems it is in R itself.
rstub
commented
6 months ago I am not sure where the UB should be or where the abort is coming from. Does the file dqrng.Rcheck/tests/testthat.Rout contain anything that could indicate which test is failing?
barracuda156
commented
6 months ago @rstub Sorry, been busy with other stuff, I rerun tests and respond here tomorrow.
barracuda156
commented
6 months ago @rstub This is what I get, running tests from 6048c1b31d62d5141308c8bd1908e51bde226054 version:
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)
> library(testthat)
> library(dqrng)
>
> test_check("dqrng")
*** caught bus error ***
address 0x1ceca000, cause 'invalid alignment'
Traceback:
1: cloned_calls(stream = 0)
2: eval_bare(expr, quo_get_env(quo))
3: quasi_label(enquo(object), label, arg = "object")
4: expect_true(cloned_calls(stream = 0))
5: eval(code, test_env)
6: eval(code, test_env)
7: withCallingHandlers({ eval(code, test_env) if (!handled && !is.null(test)) { skip_empty() }}, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, message = handle_message, error = handle_error)
8: doTryCatch(return(expr), name, parentenv, handler)
9: tryCatchOne(expr, names, parentenv, handlers[[1L]])
10: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
11: doTryCatch(return(expr), name, parentenv, handler)
12: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), names[nh], parentenv, handlers[[nh]])
13: tryCatchList(expr, classes, parentenv, handlers)
14: tryCatch(withCallingHandlers({ eval(code, test_env) if (!handled && !is.null(test)) { skip_empty() }}, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, message = handle_message, error = handle_error), error = handle_fatal, skip = function(e) { })
15: test_code(desc, code, env = parent.frame(), default_reporter = local_interactive_reporter())
16: test_that("cloned external PCG64 gives different result", { dqrng::dqRNGkind("PCG64") dqset.seed(use_seed) expect_true(cloned_calls(stream = 0)) expect_false(cloned_calls(stream = 1)) dqrng::dqRNGkind("default")})
17: eval(code, test_env)
18: eval(code, test_env)
19: withCallingHandlers({ eval(code, test_env) if (!handled && !is.null(test)) { skip_empty() }}, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, message = handle_message, error = handle_error)
20: doTryCatch(return(expr), name, parentenv, handler)
21: tryCatchOne(expr, names, parentenv, handlers[[1L]])
22: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
23: doTryCatch(return(expr), name, parentenv, handler)
24: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), names[nh], parentenv, handlers[[nh]])
25: tryCatchList(expr, classes, parentenv, handlers)
26: tryCatch(withCallingHandlers({ eval(code, test_env) if (!handled && !is.null(test)) { skip_empty() }}, expectation = handle_expectation, skip = handle_skip, warning = handle_warning, message = handle_message, error = handle_error), error = handle_fatal, skip = function(e) { })
27: test_code(test = NULL, code = exprs, env = env, default_reporter = StopReporter$new())
28: source_file(path, env = env(env), desc = desc, error_call = error_call)
29: FUN(X[[i]], ...)
30: lapply(test_paths, test_one_file, env = env, desc = desc, error_call = error_call)
31: doTryCatch(return(expr), name, parentenv, handler)
32: tryCatchOne(expr, names, parentenv, handlers[[1L]])
33: tryCatchList(expr, classes, parentenv, handlers)
34: tryCatch(code, testthat_abort_reporter = function(cnd) { cat(conditionMessage(cnd), "\n") NULL})
35: with_reporter(reporters$multi, lapply(test_paths, test_one_file, env = env, desc = desc, error_call = error_call))
36: test_files_serial(test_dir = test_dir, test_package = test_package, test_paths = test_paths, load_helpers = load_helpers, reporter = reporter, env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, desc = desc, load_package = load_package, error_call = error_call)
37: test_files(test_dir = path, test_paths = test_paths, test_package = package, reporter = reporter, load_helpers = load_helpers, env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, load_package = load_package, parallel = parallel)
38: test_dir("testthat", package = package, reporter = reporter, ..., load_package = "installed")
39: test_check("dqrng")
An irrecoverable exception occurred. R is aborting now ...Thanks @barracuda156! I have been able to run some tests on an old i686 Laptop, which does show some of the issues you are seeing:
abort is used, probably because of dqrng::generator<pcg64>(seed) in dqRNGkind().It does not show the invalid alignment / bus error, though. Can you install the package in a test environment and create a file cloned_calls.cpp somewhere:
// cloned_calls.cpp
// [[Rcpp::plugins(cpp11)]]
#include <Rcpp.h>
// [[Rcpp::depends(dqrng,BH)]]
#include <dqrng.h>
#include <dqrng_distribution.h>
// [[Rcpp::export]]
bool cloned_calls(int stream) {
dqrng::random_64bit_accessor engine{};
auto cloned_engine = engine.clone(stream);
Rcpp::NumericVector u1(10);
Rcpp::NumericVector u2(10);
engine.generate<dqrng::uniform_distribution>(u1);
cloned_engine->generate<dqrng::uniform_distribution>(u2);
return Rcpp::is_true(Rcpp::all(u1 == u2));
}
/***R
cloned_calls(stream = 0)
cloned_calls(stream = 1)
dqrng::dqRNGkind("PCG64")
cloned_calls(stream = 0)
cloned_calls(stream = 1)
*/Then call Rscript -e "Rcpp::sourceCpp('cloned_calls.cpp')" from the directory where you created the file. Thank you!
barracuda156
commented
6 months ago @rstub I get a segfault on that:
36-244% Rscript -e "Rcpp::sourceCpp('cloned_calls.cpp')"
> cat("change RNG")
change RNG
> dqrng::dqRNGkind("PCG64")
> cat("stream = 0")
stream = 0
> cloned_calls(stream = 0)
*** caught segfault ***
address 0x556a0000, cause 'memory not mapped'
Traceback:
1: .Call(<pointer: 0xad3ac2c>, stream)
2: cloned_calls(stream = 0)
3: eval(ei, envir)
4: eval(ei, envir)
5: withVisible(eval(ei, envir))
6: source(file = srcConn, local = env, echo = echo)
7: Rcpp::sourceCpp("cloned_calls.cpp")
An irrecoverable exception occurred. R is aborting now ...
zsh: segmentation fault Rscript -e "Rcpp::sourceCpp('cloned_calls.cpp')"Do you know how to get more information from that segfault using a debugger? You can start this in gdb using R -d gdb -e "Rcpp::sourceCpp('cloned_calls.cpp')". If you are not familiar with gdb: type r<enter> at the first prompt and bt<enter> after the segfault.
rstub
commented
6 months ago One more thing. Could you compile and run this C++ program that only uses PCG:
#include <iostream>
#include <sstream>
#include <vector>
#include <pcg_random.hpp>
int main(void) {
pcg64 rng;
std::cout << rng << std::endl;
rng.set_stream(1234);
std::cout << rng << std::endl;
pcg_extras::pcg128_t number;
std::vector<pcg_extras::pcg128_t> state;
std::stringstream iss;
iss << rng;
using pcg_extras::operator>>;
while (iss >> number)
state.push_back(number);
// state[1] is the current stream
// PCG will do 2*stream + 1 to make sure stream is odd; need to revert that here
rng.set_stream(state[1]/pcg_extras::pcg128_t(2) + pcg_extras::pcg128_t(0ULL));
std::cout << rng << std::endl;
return 0;
}Do you get similar errors?
barracuda156
commented
6 months ago Do I need some specific version?
36-244% Rscript -e "Rcpp::sourceCpp('test_dqrng2.cpp')"
using C++ compiler: ‘g++-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0’
using SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’
/opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include" -I"/Users/svacchanda/Desktop/_opt_PPCSnowLeopardPorts_lang_guile2/test" -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c test_dqrng2.cpp -o test_dqrng2.o
test_dqrng2.cpp:4:10: fatal error: pcg_random.hpp: No such file or directory
4 | #include <pcg_random.hpp>
| ^~~~~~~~~~~~~~~~
compilation terminated.
make: *** [test_dqrng2.o] Error 1
Warning message:
In system2("xcrun", "--show-sdk-path", TRUE, TRUE) :
running command ''xcrun' --show-sdk-path 2>&1' had status 64
Error in Rcpp::sourceCpp("test_dqrng2.cpp") :
Error 1 occurred building shared library.
Execution haltedI do not know where this header is supposed to come from. (But since I build this manually and not in MacPorts, environment may not know about other R packages, unless those are explicitly specified.)
rstub
commented
6 months ago Sorry for being unclear. This should be compiled w/o using R at all, i.e.
g++ -I<path/to/include> -o dqrng_test2 dqrng_trst2.cpp.
Sergey Fedorov @.***> schrieb am Mo. 20. Mai 2024 um 12:06:
Do I need some specific version?
36-244% Rscript -e "Rcpp::sourceCpp('test_dqrng2.cpp')" using C++ compiler: ‘g++-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0’ using SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’ /opt/local/bin/g++-mp-13 -std=gnu++17 -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/Rcpp/include" -I"/Users/svacchanda/Desktop/_opt_PPCSnowLeopardPorts_lang_guile2/test" -isystem/opt/local/include/LegacySupport -I/opt/local/include -fPIC -pipe -Os -arch ppc -c test_dqrng2.cpp -o test_dqrng2.o test_dqrng2.cpp:4:10: fatal error: pcg_random.hpp: No such file or directory 4 | #include
| ^ ~~~compilation terminated. make: *** [test_dqrng2.o] Error 1 Warning message: In system2("xcrun", "--show-sdk-path", TRUE, TRUE) : running command ''xcrun' --show-sdk-path 2>&1' had status 64 Error in Rcpp::sourceCpp("test_dqrng2.cpp") : Error 1 occurred building shared library. Execution haltedI do not know where this header is supposed to come from. (But since I build this manually and not in MacPorts, environment may not know about other R packages, unless those are explicitly specified.)
— Reply to this email directly, view it on GitHub https://github.com/daqana/dqrng/issues/88#issuecomment-2120119464, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGUIIS2ZADMEE52GVRZ6XL3ZDHDJZAVCNFSM6AAAAABHZFYAZSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRQGEYTSNBWGQ . You are receiving this because you were mentioned.Message ID: @.***>
rstub
commented
6 months ago @barracuda156 Please let me know if you have questions concerning my above requests. I have time until next week before I have to fix the UBSAN error found by CRAN. I would like to get fixes for your use case in that release as well.
barracuda156
commented
6 months ago @rstub I will deal with this by tomorrow (will try doing it today). Thank you!
barracuda156
commented
6 months ago @rstub Is this of any help?
36-244% /opt/local/bin/g++-mp-13 -arch ppc -I/opt/local/Library/Frameworks/R.framework/Versions/4.4/Resources/library/dqrng/include -o test_dqrng2 test_dqrng2.cpp
36-244% ./test_dqrng2
zsh: segmentation fault ./test_dqrng2
36-244% sudo gdb ./test_dqrng2
Password:
GNU gdb 6.3.50-20050815 (Apple version gdb-1314.1) (Sun Nov 16 22:48:12 UTC 2008)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...Reading symbols for shared libraries ..... done
(gdb) run
Starting program: /Users/svacchanda/Desktop/_opt_PPCSnowLeopardPorts_lang_guile2/test/test_dqrng2
Reading symbols for shared libraries ++++.. done
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x6d7ffff7
0x0000391c in pcg_extras::operator<< <unsigned int, unsigned long long> ()
(gdb) bt
#0 0x0000391c in pcg_extras::operator<< <unsigned int, unsigned long long> ()
#1 0x0000456c in pcg_extras::divmod<unsigned int, unsigned long long> ()
#2 0x017fffeb in ?? ()
(gdb) disassemble $pc-0x10 $pc+0x10
Dump of assembler code from 0x390c to 0x392c:
0x0000390c <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+172>: lbz r2,29(r30)
0x00003910 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+176>: lwz r9,92(r30)
0x00003914 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+180>: rlwinm r2,r2,2,0,29
0x00003918 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+184>: add r2,r9,r2
0x0000391c <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+188>: lwz r9,0(r2)
0x00003920 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+192>: lbz r2,35(r30)
0x00003924 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+196>: slw r10,r9,r2
0x00003928 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+200>: lbz r2,28(r30)
End of assembler dump.
(gdb) info registers
r0 0x4554 17748
r1 0xbffff450 3221222480
r2 0x6d7ffff7 1837105143
r3 0xbffff52c 3221222700
r4 0x6d7fffeb 1837105131
r5 0xb9 185
r6 0x4385df64 1132846948
r7 0x9fccf645 2681009733
r8 0x2360ed05 593554693
r9 0x6d7fffeb 1837105131
r10 0xbffff51c 3221222684
r11 0x71d8 29144
r12 0x3860 14432
r13 0x0 0
r14 0x0 0
r15 0x0 0
r16 0x0 0
r17 0x0 0
r18 0x0 0
r19 0x0 0
r20 0x0 0
r21 0x0 0
r22 0x0 0
r23 0x0 0
r24 0x0 0
r25 0x0 0
r26 0xbffff970 3221223792
r27 0x8 8
r28 0x0 0
r29 0x34efa0 3469216
r30 0xbffff450 3221222480
r31 0x1ff8 8184
pc 0x391c 14620
ps 0x100000000200f030 1152921504640462896
cr 0x44000442 1140851778
lr 0x456c 17772
ctr 0x3860 14432
xer 0x0 0
mq 0x0 0
fpscr 0x0 0
vscr 0x10000 65536
vrsave 0x0 0It looks like as if the error is happening right away even before the problematic code. Could you try the following program which adds more output in the beginning?
#include <iostream>
#include <sstream>
#include <vector>
#include <pcg_random.hpp>
int main(void) {
pcg64 rng;
std::cout << "RNG created" << std::endl;
std::cout << rng << std::endl;
rng.set_stream(1234);
std::cout << "stream set" << std::endl;
std::cout << rng << std::endl;
pcg_extras::pcg128_t number;
std::vector<pcg_extras::pcg128_t> state;
std::stringstream iss;
iss << rng;
using pcg_extras::operator>>;
while (iss >> number)
state.push_back(number);
// state[1] is the current stream
// PCG will do 2*stream + 1 to make sure stream is odd; need to revert that here
rng.set_stream(state[1]/pcg_extras::pcg128_t(2) + pcg_extras::pcg128_t(0ULL));
std::cout << rng << std::endl;
return 0;
}Also, if this is indeed already a problem with printing the RNG, you could try this with the headers from https://github.com/imneme/pcg-cpp/pull/92. Thanks!
barracuda156
commented
5 months ago For the updated program, but with existing headers from dqrng:
36-244% ./test_dqrng3
RNG created
zsh: segmentation fault ./test_dqrng3
36-244% sudo gdb ./test_dqrng3
Password:
GNU gdb 6.3.50-20050815 (Apple version gdb-1314.1) (Sun Nov 16 22:48:12 UTC 2008)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...Reading symbols for shared libraries ..... done
(gdb) run
Starting program: /Users/svacchanda/Desktop/_opt_PPCSnowLeopardPorts_lang_guile2/test/test_dqrng3
Reading symbols for shared libraries ++++.. done
RNG created
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x6d7ffff7
0x00003904 in pcg_extras::operator<< <unsigned int, unsigned long long> ()
(gdb) bt
#0 0x00003904 in pcg_extras::operator<< <unsigned int, unsigned long long> ()
#1 0x00004554 in pcg_extras::divmod<unsigned int, unsigned long long> ()
#2 0x017fffeb in ?? ()
Cannot access memory at address 0x177fffea
Cannot access memory at address 0x177ffff2
(gdb) disassemble $pc-0x10 $pc+0x10
Dump of assembler code from 0x38f4 to 0x3914:
0x000038f4 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+172>: lbz r2,29(r30)
0x000038f8 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+176>: lwz r9,92(r30)
0x000038fc <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+180>: rlwinm r2,r2,2,0,29
0x00003900 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+184>: add r2,r9,r2
0x00003904 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+188>: lwz r9,0(r2)
0x00003908 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+192>: lbz r2,35(r30)
0x0000390c <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+196>: slw r10,r9,r2
0x00003910 <_ZN10pcg_extraslsIjyEENS_7uint_x4IT_T0_EERKS4_h+200>: lbz r2,28(r30)
End of assembler dump.
(gdb) info registers
r0 0x453c 17724
r1 0xbffff450 3221222480
r2 0x6d7ffff7 1837105143
r3 0xbffff52c 3221222700
r4 0x6d7fffeb 1837105131
r5 0xb9 185
r6 0x4385df64 1132846948
r7 0x9fccf645 2681009733
r8 0x2360ed05 593554693
r9 0x6d7fffeb 1837105131
r10 0xbffff51c 3221222684
r11 0x71d4 29140
r12 0x3848 14408
r13 0x0 0
r14 0x0 0
r15 0x0 0
r16 0x0 0
r17 0x0 0
r18 0x0 0
r19 0x0 0
r20 0x0 0
r21 0x0 0
r22 0x0 0
r23 0x0 0
r24 0x0 0
r25 0x0 0
r26 0xbffff970 3221223792
r27 0x8 8
r28 0x0 0
r29 0x34efa0 3469216
r30 0xbffff450 3221222480
r31 0x1f90 8080
pc 0x3904 14596
ps 0x100000000200d030 1152921504640454704
cr 0x44000442 1140851778
lr 0x4554 17748
ctr 0x3848 14408
xer 0x0 0
mq 0x0 0
fpscr 0x0 0
vscr 0x10000 65536
vrsave 0x0 0Thanks! So it is indeed already the printing of the RNG that is problematic. The latest version of PCG does have some fixes concerning printing. It would be great if you could test against that version.
barracuda156
commented
5 months ago Did I understand correctly, that I can just replace those 3 headers borrowing new versions from https://github.com/imneme/pcg-cpp/tree/master/include and apply a patch from https://github.com/imneme/pcg-cpp/pull/92 onto pcg_random.hpp?
I did that now, it still fails:
36-244% ./test_dqrng3
RNG created
zsh: segmentation fault ./test_dqrng3
36-244% sudo gdb ./test_dqrng3
Password:
GNU gdb 6.3.50-20050815 (Apple version gdb-1314.1) (Sun Nov 16 22:48:12 UTC 2008)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...Reading symbols for shared libraries ..... done
(gdb) run
Starting program: /Users/svacchanda/Desktop/_opt_PPCSnowLeopardPorts_lang_guile2/test/test_dqrng3
Reading symbols for shared libraries ++++.. done
RNG created
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x077ffff7
0x00003970 in pcg_extras::operator<< <unsigned int, unsigned long long> ()
(gdb) bt
#0 0x00003970 in pcg_extras::operator<< <unsigned int, unsigned long long> ()
#1 0x0000457c in pcg_extras::divmod<unsigned int, unsigned long long> ()
#2 0x017fffeb in ?? ()
Cannot access memory at address 0x177fffea
Cannot access memory at address 0x177ffff2Thanks! You understood correctly. This means that even the most recent version of PCG does fail in your environment. I am not sure I will be able to fix this. It would be good to get an issue created in the PCG repo, even though that is pretty quiet right now. I can provide suitable code for this, but will need your help to create the actual error messages. Would you be interested in working on this?
As for this package: I am considering disabling PCG completely or at least the problematic parts. Do you know if there is a good way to identify your platform at compile time?
barracuda156
commented
5 months ago I can provide suitable code for this, but will need your help to create the actual error messages. Would you be interested in working on this?
Yes, sure. This is not something I can fix myself, but I can test whatever needed.
As for this package: I am considering disabling PCG completely or at least the problematic parts. Do you know if there is a good way to identify your platform at compile time?
If we can use macros inside C/C++ code, then this gonna work:
#if !(defined(__APPLE__) && defined(__POWERPC__))
"some code to be excluded on macOS powerpc"
#endifThis assumes the issue applies to ppc and ppc64, but only on macOS. (I have no way presently to test anything on ppc64 or on BSD, so we can guess here.)
For Linux and BSD, I think, __powerpc__ is used, but not really sure here. Perhaps until someone comes up to complain that it is broken on their platform, it makes a better sense to consider only macOS powerpc (unless we know the issue is related to endianness, in which case it will obviously affect every other big-endian platform).
rstub
commented
5 months ago In #91 I am now excluding PCG completely on MacOS for PowerPC, but now I have to exclude several tests to run as well. Can you send me the output of Rscript -e "Sys.info(); R.version" on your system?
barracuda156
commented
5 months ago 36-231% Rscript -e "Sys.info(); R.version"
sysname
"Darwin"
release
"10.0.0d2"
version
"Darwin Kernel Version 10.0.0d2: Fri Oct 10 19:37:52 PDT 2008; root:xnu-1346.7~1/RELEASE_PPC"
nodename
"36-231.wireless-city.scu.edu.tw"
machine
"Power Macintosh"
login
"svacchanda"
user
"svacchanda"
effective_user
"svacchanda"
_
platform powerpc-apple-darwin10.0.0d2
arch powerpc
os darwin10.0.0d2
system powerpc, darwin10.0.0d2
status
major 4
minor 4.0
year 2024
month 04
day 24
svn rev 86474
language R
version.string R version 4.4.0 (2024-04-24)
nickname Puppy CupPossible values for the triple will range from powerpc-apple-darwin8.* to powerpc-apple-darwin10.*, the latest possible being powerpc-apple-darwin10.8.0 (= 10.6.8).
rstub
commented
5 months ago Can you give the version from #91 a try? Everything but the UB from xoshiro.h should be fixed in that version.
barracuda156
commented
5 months ago I have applied three commits as patches on top of the master, and ran tests:
---> Testing R-dqrng
Executing: cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng-e55775f6826be353bc0119e50227079a406faded" && /opt/local/bin/R CMD check ./dqrng_0.4.0.1.tar.gz --no-manual --no-build-vignettes
* using log directory ‘/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-dqrng/R-dqrng/work/dqrng-e55775f6826be353bc0119e50227079a406faded/dqrng.Rcheck’
* using R version 4.4.0 (2024-04-24)
* using platform: powerpc-apple-darwin10.0.0d2 (32-bit)
* R was compiled by
gcc-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0
GNU Fortran (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0
* running under: OS X Snow Leopard 10.6
* using session charset: UTF-8
* using options ‘--no-manual --no-build-vignettes’
* checking for file ‘dqrng/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘dqrng’ version ‘0.4.0.1’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘dqrng’ can be installed ... OK
* used C++ compiler: ‘g++-mp-13 (MacPorts gcc13 13.2.0_4+stdlib_flag) 13.2.0’
* used SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’
* checking installed package size ... OK
* checking package directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking whether startup messages can be suppressed ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in shell scripts ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking compiled code ... OK
* checking files in ‘vignettes’ ... WARNING
Files in the 'vignettes' directory but no files in 'inst/doc':
‘cpp-api.Rmd’ ‘data/no-replacement-high.RDS’
‘data/no-replacement-long.RDS’ ‘data/no-replacement-low.RDS’
‘data/no-replacement-medium.RDS’ ‘data/replacement.RDS’ ‘dqrng.Rmd’
‘parallel.Rmd’ ‘sample.Rmd’
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
Running ‘testthat.R’
ERROR
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
`expected`: TRUE
── Failure ('test-cpp.R:63:3'): long_jump() for xoshiro256++ works ─────────────
xoshiro_plus_plus_long_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
── Failure ('test-cpp.R:67:3'): long_jump() for xoshiro256** works ─────────────
xoshiro_star_star_long_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
[ FAIL 6 | WARN 0 | SKIP 10 | PASS 157 ]
Error: Test failures
Execution halted
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... WARNING
Directory 'inst/doc' does not exist.
Package vignettes without corresponding single PDF/HTML:
‘cpp-api.Rmd’
‘dqrng.Rmd’
‘parallel.Rmd’
‘sample.Rmd’
* checking running R code from vignettes ...
‘cpp-api.Rmd’ using ‘UTF-8’... OK
‘dqrng.Rmd’ using ‘UTF-8’... OK
‘parallel.Rmd’ using ‘UTF-8’... OK
‘sample.Rmd’ using ‘UTF-8’... OK
OK
* checking re-building of vignette outputs ... SKIPPED
* DONE
Status: 1 ERROR, 2 WARNINGs
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(testthat)
> library(dqrng)
>
> test_check("dqrng")
[ FAIL 6 | WARN 0 | SKIP 10 | PASS 157 ]
══ Skipped tests (10) ══════════════════════════════════════════════════════════
• No long-vector support (5): 'test-sample.R:74:5', 'test-sample.R:86:5',
'test-sample.R:98:5', 'test-sample.R:110:5', 'test-sample.R:139:5'
• powerpc_apple is TRUE (5): 'test-external-generator.R:73:3',
'test-generators.R:43:3', 'test-generators.R:53:3', 'test-generators.R:63:3',
'test-generators.R:74:3'
══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-cpp.R:47:3'): jump() for xoroshiro128+ works ─────────────────
xoroshiro_plus_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
── Failure ('test-cpp.R:51:3'): jump() for xoroshiro128++ works ────────────────
xoroshiro_plus_plus_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
── Failure ('test-cpp.R:55:3'): jump() for xoroshiro128** works ────────────────
xoroshiro_star_star_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
── Failure ('test-cpp.R:59:3'): long_jump() for xoshiro256+ works ──────────────
xoshiro_plus_long_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
── Failure ('test-cpp.R:63:3'): long_jump() for xoshiro256++ works ─────────────
xoshiro_plus_plus_long_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
── Failure ('test-cpp.R:67:3'): long_jump() for xoshiro256** works ─────────────
xoshiro_star_star_long_jump() is not TRUE
`actual`: FALSE
`expected`: TRUE
[ FAIL 6 | WARN 0 | SKIP 10 | PASS 157 ]
Error: Test failures
Execution haltedLooks like the UB is serious enough to fail the tests. I have pushed some additional changes to #91. Can you test those, please?
barracuda156
commented
5 months ago Everything passes now:
R version 4.4.0 (2024-04-24) -- "Puppy Cup"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)
> library(testthat)
> library(dqrng)
>
> test_check("dqrng")
[ FAIL 0 | WARN 0 | SKIP 10 | PASS 163 ]
══ Skipped tests (10) ══════════════════════════════════════════════════════════
• No long-vector support (5): 'test-sample.R:74:5', 'test-sample.R:86:5',
'test-sample.R:98:5', 'test-sample.R:110:5', 'test-sample.R:139:5'
• powerpc_apple is TRUE (5): 'test-external-generator.R:73:3',
'test-generators.R:43:3', 'test-generators.R:53:3', 'test-generators.R:63:3',
'test-generators.R:74:3'
[ FAIL 0 | WARN 0 | SKIP 10 | PASS 163 ]
>
> proc.time()
user system elapsed
68.323 12.503 130.621 Thank you!
rstub
commented
5 months ago Excellent! I will send this as version 0.4.1 to CRAN. Once things have settled there I will get back to you w.r.t. creating the issue for PCG.
rstub
commented
5 months ago Hi @barracuda156 Shall we have a look at creating an issue for PCG? My suggestion would be to use
#include <iostream>
#include "pcg_random.hpp"
int main(void) {
pcg64 rng;
rng.seed(20240409);
rng.set_stream(90404202);
std::cout << rng << std::endl;
return 0;
}As example code. With upstream PCG, this should produce an error like
pcg_random.hpp:322:37: error: no match for 'operator|' (operand types are 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' and 'int')
322 | inc_ = (specific_seq << 1) | 1;With the patch from https://github.com/imneme/pcg-cpp/pull/92 applied, this should compile but produce a segmentation fault upon execution.
Do you want to submit the issue? Or do prefer providing me with the necessary output from you environment so that I can submit one?
rstub
commented
3 months ago @barracuda156, Are you interested in bring this to the PCG developers together?
barracuda156
commented
3 months ago @rstub Sure, let’s do it. What should I do from my end?
rstub
commented
3 months ago @barracuda156 I think it would be best to supply error messages from a minimal example. Could you compile
#include <iostream>
#include "pcg_random.hpp"
int main(void) {
pcg64 rng;
rng.seed(20240409);
rng.set_stream(90404202);
std::cout << rng << std::endl;
return 0;
}with upstream PCG as well as the patch from https://github.com/imneme/pcg-cpp/pull/92 applied? With the upstream version, you should get error: no match for 'operator|'. With the patch things should compile, but we expect a segfault upon execution. It would be good if you could provide the exact error messages you get in these two cases. Thanks!
barracuda156
commented
3 months ago Sorry for a delay, will do this tomorrow.
barracuda156
commented
3 months ago 36-25% /opt/local/bin/g++-mp-14 -arch ppc -I./pcg-cpp/include -o pcg_test pcg_test.cpp
In file included from pcg_test.cpp:2:
./pcg-cpp/include/pcg_random.hpp: In instantiation of 'void pcg_detail::specific_stream<itype>::set_stream(itype) [with itype = pcg_extras::uint_x4<unsigned int, long long unsigned int>]':
pcg_test.cpp:7:17: required from here
7 | rng.set_stream(90404202);
| ~~~~~~~~~~~~~~^~~~~~~~~~
./pcg-cpp/include/pcg_random.hpp:336:37: error: no match for 'operator|' (operand types are 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' and 'in')
336 | inc_ = (specific_seq << 1) | 1;
| ~~~~~~~~~~~~~~~~~~~~^~~
./pcg-cpp/include/pcg_random.hpp:336:37: note: candidate: 'operator|(int, int)' (built-in)
./pcg-cpp/include/pcg_random.hpp:336:37: note: no known conversion for argument 1 from 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' to 'int'
./pcg-cpp/include/pcg_random.hpp:336:37: note: candidate: 'operator|(long long unsigned int, int)' (built-in)
./pcg-cpp/include/pcg_random.hpp:336:37: note: no known conversion for argument 1 from 'pcg_extras::uint_x4<unsigned int, long long unsigned int>' to 'long long unsigned int'
In file included from ./pcg-cpp/include/pcg_extras.hpp:84,
from ./pcg-cpp/include/pcg_random.hpp:114:
./pcg-cpp/include/pcg_uint128.hpp:837:22: note: candidate: 'template<class UInt, class UIntX2> pcg_extras::uint_x4<U, V> pcg_extras::operator|(const uint_x4<U, V>&, const uint_x4<U, V>&)'
837 | uint_x4<UInt,UIntX2> operator|(const uint_x4<UInt,UIntX2>& a,
| ^~~~~~~~
./pcg-cpp/include/pcg_uint128.hpp:837:22: note: template argument deduction/substitution failed:
./pcg-cpp/include/pcg_random.hpp:336:37: note: mismatched types 'const pcg_extras::uint_x4<U, V>' and 'int'
336 | inc_ = (specific_seq << 1) | 1;
| ~~~~~~~~~~~~~~~~~~~~^~~
36-25% /opt/local/bin/g++-mp-14 -arch ppc -I./pcg-cpp/include -o pcg_test pcg_test.cpp
36-25% ./pcg_test
zsh: segmentation fault ./pcg_test36-25% sudo /opt/local/bin/gdb-apple ./pcg_test
Password:
GNU gdb 6.3.50.20050815-cvs (Wed Apr 13 04:55:50 UTC 2022)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "--host=powerpc-apple-darwin10.0.0d2 --target="...Reading symbols for shared libraries ..... done
(gdb) run
Starting program: /Users/svacchanda/Dev_tmp/dqrng_test_tmp/pcg_test
Reading symbols for shared libraries ++++.. done
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x037ffffa
0x00002924 in pcg_extras::operator<< <unsigned int, unsigned long long> ()
(gdb) bt
#0 0x00002924 in pcg_extras::operator<< <unsigned int, unsigned long long> ()
#1 0x000038d4 in pcg_extras::divmod<unsigned int, unsigned long long> ()
#2 0x017fffee in ?? ()
Cannot access memory at address 0x417fffee
Cannot access memory at address 0x417ffff6First version as-is (git clone https://github.com/imneme/pcg-cpp).
Second version, with segfault, is with this line changed: https://github.com/imneme/pcg-cpp/pull/92/commits/f1ecb840ca7f9b8ae377e5e87409aa1b15825a59
rstub
commented
3 months ago Thanks! I have now created https://github.com/imneme/pcg-cpp/issues/93. Let's see how that goes.