search

pola-rs / r-polars

Polars R binding
https://pola-rs.github.io/r-polars/
Other
479 stars 36 forks source link

unable to install rpolars arm64 macOS #4

Closed JosiahParry closed 1 year ago

JosiahParry commented 1 year ago

There are no arm64 binaries for rpolars. When installing from source the following error is emitted. 

remotes::install_github("pola-rs/r-polars")

> remotes::install_github("pola-rs/r-polars")
Downloading GitHub repo pola-rs/r-polars@HEAD
✔  checking for file ‘/private/var/folders/wd/xq999jjj3bx2w8cpg7lkfxlm0000gn/T/RtmptyYC8r/remotes2cb274cf5d3a/pola-rs-r-polars-92849ca/DESCRIPTION’ ...
─  preparing ‘rpolars’:
  ✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘rpolars_0.3.0.tar.gz’

* installing *source* package ‘rpolars’ ...
** using staged installation
** libs
rm -Rf rpolars.so ./rust/target/release/librpolars.a entrypoint.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c entrypoint.c -o entrypoint.o
cargo build --lib --release --manifest-path=./rust/Cargo.toml
error: no matching package named `polars-core` found
location searched: https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac
required by package `rpolars v0.1.0 (/private/var/folders/wd/xq999jjj3bx2w8cpg7lkfxlm0000gn/T/RtmpTLYPCH/R.INSTALL2e213fb31235/rpolars/src/rust)`
make: *** [rust/target/release/librpolars.a] Error 101
ERROR: compilation failed for package ‘rpolars’
* removing ‘/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/rpolars’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/rpolars’
Warning message:
  In i.p(...) :
  installation of package ‘/var/folders/wd/xq999jjj3bx2w8cpg7lkfxlm0000gn/T//RtmptyYC8r/file2cb259f1c6bf/rpolars_0.3.0.tar.gz’ had non-zero exit status

Do you have a suggestion on how to complete install?

sorhawell commented 1 year ago

Hi Josiah, thankyou for asking :)

I just list the current unknowns to me and see what we can do about it.

Binary support:

Currently extendr asfaik does not have any arm64 templates and GitHub Actions the current binary builder, do not have any arm64 arch except some self-hosted machines. At some point I hope r-polars can be built on cran, but as polars uses rustc nightly version, this is not supported by CRAN. CRAN did start supporting stable rust which is fantastic, so maybe the snowball has start rolling. For macos_x86_64 windows_x86_64 and Linux x86_64 installing from binary takes 5 secs or so, but that is very unuseful here :)

Build from source

I'd see if I can get hold on Arm64 Mac machine to test a manual build. It will likely require to install rust and rust nightly, which isn't too complicated or tedious, but I have to see if some extra compiler flags would be needed. I have asked the extendr wizards on their Discord (they have been very kind and helpful to this project). If you find anything to add to the very sparse build guide it would be very much appreciated.

sorhawell commented 1 year ago

R-universe (alternative to cran, discussion) https://github.com/r-universe-org/help/issues/97

JosiahParry commented 1 year ago

I can try seeing if there is another extendr package that compiles. I couldn't install froim source.

* installing to library ‘/Users/josiahparry/Library/Caches/org.R-project.R/R/renv/library/r-polars-f3da1b80/R-4.1/aarch64-apple-darwin20’
* installing *source* package ‘rpolars’ ...
** using staged installation
rm -Rf rpolars.so ./rust/target/release/librpolars.a entrypoint.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c entrypoint.c -o entrypoint.o
** libs
cargo build --lib --release --manifest-path=./rust/Cargo.toml
make: cargo: No such file or directory
make: *** [rust/target/release/librpolars.a] Error 1
ERROR: compilation failed for package ‘rpolars’
* removing ‘/Users/josiahparry/Library/Caches/org.R-project.R/R/renv/library/r-polars-f3da1b80/R-4.1/aarch64-apple-darwin20/rpolars’

Exited with status 1.

Forked and tried to build the rust source and ran into 

cargo build
   Compiling indexmap v1.9.1
   Compiling libz-ng-sys v1.1.8
   Compiling futures-executor v0.3.24
   Compiling parking_lot_core v0.9.4
   Compiling signal-hook v0.3.14
   Compiling miniz_oxide v0.5.4
   Compiling rayon v1.5.3
   Compiling num-bigint v0.4.3
error: failed to run custom build command for `libz-ng-sys v1.1.8`

Caused by:
  process didn't exit successfully: `/Users/josiahparry/github/r-polars/src/rust/target/debug/build/libz-ng-sys-2c76a8c403f4e888/build-script-build_zng` (exit status: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_aarch64-apple-darwin = None
  CMAKE_TOOLCHAIN_FILE_aarch64_apple_darwin = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_aarch64-apple-darwin = None
  CMAKE_GENERATOR_aarch64_apple_darwin = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_aarch64-apple-darwin = None
  CMAKE_PREFIX_PATH_aarch64_apple_darwin = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_aarch64-apple-darwin = None
  CMAKE_aarch64_apple_darwin = None
  HOST_CMAKE = None
  CMAKE = None
  running: cd "/Users/josiahparry/github/r-polars/src/rust/target/debug/build/libz-ng-sys-2047b35aaa6a57f9/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/Users/josiahparry/.cargo/registry/src/github.com-1ecc6299db9ec823/libz-ng-sys-1.1.8/src/zlib-ng" "-DCMAKE_OSX_ARCHITECTURES=arm64" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=OFF" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_INSTALL_PREFIX=/Users/josiahparry/github/r-polars/src/rust/target/debug/build/libz-ng-sys-2047b35aaa6a57f9/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -arch arm64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -arch arm64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -arch arm64" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Debug"

  --- stderr
  thread 'main' panicked at '
  failed to execute command: No such file or directory (os error 2)
  is `cmake` not installed?

  build script failed, must exit now', /Users/josiahparry/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.49/src/lib.rs:1104:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
JosiahParry commented 1 year ago

Note that cmake is installed

cmake --version
cmake version 3.25.1
sorhawell commented 1 year ago

A potential similar issue was resolved by enforcing cmake was in PATH

https://github.com/rust-lang/libz-sys/issues/106 https://github.com/rust-lang/flate2-rs/issues/318

@JosiahParry could this be helpful?

sorhawell commented 1 year ago

I have tried to get hold of a M1 machine with little success. My best option seems to be to reactivate my account at AWS and rent machine for 1$/hour or replace my ancient macbook for 3000$, Jesus they are expensive ^^

sorhawell commented 1 year ago

Potentially r-polars github actions could cross-compile and release the rust binaries for M1 or any architecture. The R package rpolars cannot be cross-compiled. However the makefile could be modified to download the binary if no cargo is found. A mac user would likely then only need Xcode to build R package rpolars, and it should only take a few seconds instead of 20 minutes of compilation.

Such a stunt was previously used by @yutannihilation https://yutani.rbind.io/post/2021-08-01-unofficial-introduction-to-extendr-appendix-i-setup-github-actions-ci-and-more/#provide-precompiled-binaries-for-windows

yutannihilation commented 1 year ago

You might need to include cargo and cmake in the PATHs by yourself. I think it's usually set in ~/.profile or something, but it seems it's not sourced when you install an R package on macOS machines (I'm usually on Windows and Linux, so I don't know well).

For example, hellorust sets PATH explicitly here.

https://github.com/r-rust/hellorust/blob/e10cfe2f6133346764321a780063956c8fcd4a0c/src/Makevars#L17

sorhawell commented 1 year ago

cross compilation

from workdir r-polars/src/rust I cross-compiled for M1 by running these lines 

rustup target add aarch64-apple-darwin --toolchain nightly
rustup default nightly-aarch64-apple-darwin
cargo build --lib --release

then I uploaded the binary librpolars.a to here

Installation on M1 without rust compilation

Clone r-polars repo and remember to checkout arm64 branch (I forgot twice myself ^^)

download and copy above mentioned binary into r-polars/src/rust/target/release/librpolars.a the Makevars file in arm64 branch do not run cargo but expect the binary to already be there.

either install in dev environment

start R session with repo root as workdir

renv::restore() #should set all suggested packages
rextendr::document()
devtools::test() # to run all unit tests

or without installing any packages, just from any R installation >=4.1.0

from terminal with repo root as workdir 

R CMD INSTALL --preclean --no-multiarch --with-keep.source .

simple hello world

library(rpolars)
pl$DataFrame(iris)

polars DataFrame: shape: (150, 5)
┌──────────────┬─────────────┬──────────────┬─────────────┬───────────┐
│ Sepal.Length ┆ Sepal.Width ┆ Petal.Length ┆ Petal.Width ┆ Species   │
│ ---          ┆ ---         ┆ ---          ┆ ---         ┆ ---       │
│ f64          ┆ f64         ┆ f64          ┆ f64         ┆ cat       │
╞══════════════╪═════════════╪══════════════╪═════════════╪═══════════╡
│ 5.1          ┆ 3.5         ┆ 1.4          ┆ 0.2         ┆ setosa    │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ 4.9          ┆ 3.0         ┆ 1.4          ┆ 0.2         ┆ setosa    │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ 4.7          ┆ 3.2         ┆ 1.3          ┆ 0.2         ┆ setosa    │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ 4.6          ┆ 3.1         ┆ 1.5          ┆ 0.2         ┆ setosa    │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ ...          ┆ ...         ┆ ...          ┆ ...         ┆ ...       │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ 6.3          ┆ 2.5         ┆ 5.0          ┆ 1.9         ┆ virginica │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ 6.5          ┆ 3.0         ┆ 5.2          ┆ 2.0         ┆ virginica │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ 6.2          ┆ 3.4         ┆ 5.4          ┆ 2.3         ┆ virginica │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤
│ 5.9          ┆ 3.0         ┆ 5.1          ┆ 1.8         ┆ virginica │
└──────────────┴─────────────┴──────────────┴─────────────┴───────────┘
sorhawell commented 1 year ago

@JosiahParry I would love, if you would also try out "Installation on M1 without rust compilation" as above. If it works, we are quite close for an easy installation supported for M1 macbooks. The makevars file could download the binary automatically.

I tested it for my own nightly-x86_64-apple-darwin arch, it should likely also work for nightly-aarch64-apple-darwin

DyfanJones commented 1 year ago

Hi @sorhawell and @JosiahParry I don't know if this helps but these are the steps I took to get r-polars working on my M1.

# terminal setup
# https://rustup.rs/
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# used default option 1

rustup toolchain install nightly
rustup target add aarch64-apple-darwin --toolchain nightly
rustup default nightly-aarch64-apple-darwin
# install r-polars from R
Sys.getenv("PATH") |> grepl("cargo/bin", x= _)
sessioninfo::session_info()
remotes::install_github("pola-rs/r-polars")

Output from R commands:

> Sys.getenv("PATH") |> grepl("cargo/bin", x= _)
[1] TRUE
> sessioninfo::session_info()
─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.2.2 (2022-10-31)
 os       macOS Ventura 13.0.1
 system   aarch64, darwin20
 ui       RStudio
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       Europe/London
 date     2023-01-05
 rstudio  2022.12.0+353 Elsbeth Geranium (desktop)
 pandoc   2.19.2 @ /opt/homebrew/bin/pandoc

─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 package     * version date (UTC) lib source
 cli           3.5.0   2022-12-20 [1] CRAN (R 4.2.0)
 rstudioapi    0.14    2022-08-22 [1] CRAN (R 4.2.0)
 sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.2.0)

 [1] /Users/dyfanjones/Library/R/arm64/4.2/library
 [2] /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> remotes::install_github("pola-rs/r-polars")
Downloading GitHub repo pola-rs/r-polars@HEAD
── R CMD build ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/sp/scbzkbwx6hbchmylsx0y52k80000gn/T/RtmpeouknQ/remotes12ea20ed60c6/pola-rs-r-polars-92849ca/DESCRIPTION’ ...
─  preparing ‘rpolars’:
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘rpolars_0.3.0.tar.gz’

Installing package into ‘/Users/dyfanjones/Library/R/arm64/4.2/library’
(as ‘lib’ is unspecified)
* installing *source* package ‘rpolars’ ...
** using staged installation
** libs
rm -Rf rpolars.so ./rust/target/release/librpolars.a entrypoint.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c entrypoint.c -o entrypoint.o
cargo build --lib --release --manifest-path=./rust/Cargo.toml
    Updating git repository `https://github.com/extendr/extendr`
    Updating crates.io index
    Updating git repository `https://github.com/pola-rs/polars.git`
    Updating git repository `https://github.com/ritchie46/arrow2`
    Updating git submodule `https://github.com/apache/arrow-testing`
    Updating git submodule `https://github.com/apache/parquet-testing`
    Updating git repository `https://github.com/ritchie46/jsonpath`
 Downloading crates ...
  Downloaded adler32 v1.2.0
  Downloaded ahash v0.8.1
  Downloaded alloc-stdlib v0.2.2
  Downloaded alloc-no-stdlib v2.0.4
  Downloaded async-trait v0.1.58
  Downloaded streaming-iterator v0.1.8
  Downloaded cfg-if v1.0.0
  Downloaded comfy-table v6.1.2
  Downloaded crc-catalog v1.1.1
  Downloaded cmake v0.1.49
  Downloaded async-stream-impl v0.3.3
  Downloaded const-random-macro v0.1.14
  Downloaded const-random v0.1.14
  Downloaded cc v1.0.73
  Downloaded streaming-decompression v0.1.2
  Downloaded avro-schema v0.3.0
  Downloaded aho-corasick v0.7.19
  Downloaded strength_reduce v0.2.3
  Downloaded pin-project v1.0.12
  Downloaded dirs v4.0.0
  Downloaded tiny-keccak v2.0.2
  Downloaded strum_macros v0.24.3
  Downloaded version_check v0.9.4
  Downloaded dirs-sys v0.3.7
  Downloaded float-cmp v0.9.0
  Downloaded crossbeam-utils v0.8.11
  Downloaded futures-sink v0.3.24
  Downloaded futures-io v0.3.25
  Downloaded futures-core v0.3.24
  Downloaded futures-task v0.3.25
  Downloaded dyn-clone v1.0.9
  Downloaded either v1.8.0
  Downloaded thiserror v1.0.37
  Downloaded crc v2.1.0
  Downloaded futures-macro v0.3.24
  Downloaded foreign_vec v0.1.0
  Downloaded futures v0.3.24
  Downloaded hash_hasher v2.0.3
  Downloaded halfbrown v0.1.15
  Downloaded flate2 v1.0.24
  Downloaded ethnum v1.3.0
  Downloaded enum_dispatch v0.3.8
  Downloaded lexical-core v0.8.5
  Downloaded lexical v6.1.1
  Downloaded json-deserializer v0.4.4
  Downloaded itoa v1.0.3
  Downloaded hashbrown v0.13.1
  Downloaded lexical-parse-float v0.8.5
  Downloaded value-trait v0.5.1
  Downloaded syn v1.0.99
  Downloaded lexical-write-integer v0.8.5
  Downloaded lexical-parse-integer v0.8.6
  Downloaded zstd-safe v5.0.2+zstd.1.5.2
  Downloaded lexical-write-float v0.8.5
  Downloaded crossterm v0.25.0
  Downloaded libm v0.2.5
  Downloaded libflate_lz77 v1.1.0
  Downloaded crc32fast v1.3.2
  Downloaded strum v0.24.1
  Downloaded crossbeam-deque v0.8.2
  Downloaded core-foundation-sys v0.8.3
  Downloaded bytemuck v1.12.1
  Downloaded time v0.1.44
  Downloaded thiserror-impl v1.0.37
  Downloaded unicode-width v0.1.10
  Downloaded xxhash-rust v0.8.6
  Downloaded lock_api v0.4.7
  Downloaded futures-executor v0.3.24
  Downloaded flume v0.10.14
  Downloaded fallible-streaming-iterator v0.1.9
  Downloaded futures-channel v0.3.24
  Downloaded futures-util v0.3.24
  Downloaded lazy_static v1.4.0
  Downloaded log v0.4.17
  Downloaded indexmap v1.9.1
  Downloaded multiversion v0.6.1
  Downloaded lz4 v1.24.0
  Downloaded num v0.4.0
  Downloaded now v0.1.3
  Downloaded parking_lot_core v0.9.4
  Downloaded lexical-util v0.8.5
  Downloaded parquet-format-safe v0.2.4
  Downloaded hashbrown v0.12.3
  Downloaded jobserver v0.1.25
  Downloaded zstd-sys v2.0.1+zstd.1.5.2
  Downloaded array-init-cursor v0.2.0
  Downloaded ahash v0.7.6
  Downloaded adler v1.0.2
  Downloaded iana-time-zone v0.1.53
  Downloaded hex v0.4.3
  Downloaded heck v0.4.0
  Downloaded arrow-format v0.8.1
  Downloaded bitflags v1.3.2
  Downloaded base64 v0.13.1
  Downloaded async-stream v0.3.3
  Downloaded bytemuck_derive v1.2.1
  Downloaded glob v0.3.0
  Downloaded memoffset v0.6.5
  Downloaded num_cpus v1.13.1
  Downloaded num-iter v0.1.43
  Downloaded num-traits v0.2.15
  Downloaded parking_lot v0.12.1
  Downloaded mio v0.8.5
  Downloaded mimalloc v0.1.32
  Downloaded memmap2 v0.5.7
  Downloaded once_cell v1.13.1
  Downloaded num-complex v0.4.2
  Downloaded memchr v2.5.0
  Downloaded num-bigint v0.4.3
  Downloaded byteorder v1.4.3
  Downloaded num-integer v0.1.45
  Downloaded libflate v1.2.0
  Downloaded fxhash v0.2.1
  Downloaded getrandom v0.2.7
  Downloaded bytes v1.3.0
  Downloaded num-rational v0.4.1
  Downloaded nanorand v0.7.0
  Downloaded miniz_oxide v0.5.4
  Downloaded brotli-decompressor v2.3.2
  Downloaded matrixmultiply v0.3.2
  Downloaded crossbeam-epoch v0.9.10
  Downloaded regex v1.6.0
  Downloaded signal-hook-mio v0.2.3
  Downloaded signal-hook v0.3.14
  Downloaded scopeguard v1.1.0
  Downloaded multiversion-macros v0.6.1
  Downloaded crossbeam-channel v0.5.6
  Downloaded autocfg v1.1.0
  Downloaded serde_derive v1.0.144
  Downloaded slab v0.4.7
  Downloaded anyhow v1.0.65
  Downloaded paste v1.0.7
  Downloaded pin-project-internal v1.0.12
  Downloaded unicode-ident v1.0.1
  Downloaded zstd v0.11.2+zstd.1.5.2
  Downloaded pin-utils v0.1.0
  Downloaded ppv-lite86 v0.2.16
  Downloaded chrono v0.4.22
  Downloaded proc-macro-hack v0.5.20+deprecated
  Downloaded planus v0.3.1
  Downloaded rand_distr v0.4.3
  Downloaded ndarray v0.15.6
  Downloaded parquet2 v0.17.1
  Downloaded rle-decode-fast v1.0.3
  Downloaded libc v0.2.132
  Downloaded rand_core v0.6.4
  Downloaded rand v0.8.5
  Downloaded rand_chacha v0.3.1
  Downloaded pin-project-lite v0.2.9
  Downloaded libR-sys v0.3.0
  Downloaded rayon v1.5.3
  Downloaded lz4-sys v1.9.4
  Downloaded libmimalloc-sys v0.1.28
  Downloaded ryu v1.0.11
  Downloaded seq-macro v0.3.1
  Downloaded rayon-core v1.9.3
  Downloaded regex-syntax v0.6.27
  Downloaded serde v1.0.144
  Downloaded serde_json v1.0.85
  Downloaded libz-ng-sys v1.1.8
  Downloaded state v0.5.3
  Downloaded snap v1.0.5
  Downloaded proc-macro2 v1.0.40
  Downloaded rawpointer v0.2.1
  Downloaded crunchy v0.2.2
  Downloaded smallvec v1.9.0
  Downloaded simdutf8 v0.1.4
  Downloaded rustc_version v0.4.0
  Downloaded quote v1.0.20
  Downloaded signal-hook-registry v1.4.0
  Downloaded simd-json v0.7.0
  Downloaded static_assertions v1.1.0
  Downloaded spin v0.9.4
  Downloaded smartstring v1.0.1
  Downloaded rustversion v1.0.9
  Downloaded semver v1.0.16
  Downloaded brotli v3.3.4
   Compiling libc v0.2.132
   Compiling autocfg v1.1.0
   Compiling proc-macro2 v1.0.40
   Compiling cfg-if v1.0.0
   Compiling unicode-ident v1.0.1
   Compiling quote v1.0.20
   Compiling syn v1.0.99
   Compiling version_check v0.9.4
   Compiling once_cell v1.13.1
   Compiling libm v0.2.5
   Compiling serde_derive v1.0.144
   Compiling serde v1.0.144
   Compiling ahash v0.7.6
   Compiling num-traits v0.2.15
   Compiling scopeguard v1.1.0
   Compiling futures-core v0.3.24
   Compiling num-integer v0.1.45
   Compiling memchr v2.5.0
   Compiling static_assertions v1.1.0
   Compiling getrandom v0.2.7
   Compiling jobserver v0.1.25
   Compiling crossbeam-utils v0.8.11
   Compiling lexical-util v0.8.5
   Compiling cc v1.0.73
   Compiling hashbrown v0.12.3
   Compiling indexmap v1.9.1
   Compiling slab v0.4.7
   Compiling memoffset v0.6.5
   Compiling futures-channel v0.3.24
   Compiling futures-task v0.3.25
   Compiling futures-sink v0.3.24
   Compiling crossbeam-epoch v0.9.10
   Compiling crc32fast v1.3.2
   Compiling zstd-sys v2.0.1+zstd.1.5.2
   Compiling futures-util v0.3.24
   Compiling cmake v0.1.49
   Compiling lock_api v0.4.7
   Compiling pin-utils v0.1.0
   Compiling semver v1.0.16
   Compiling log v0.4.17
   Compiling ryu v1.0.11
   Compiling itoa v1.0.3
   Compiling pin-project-lite v0.2.9
   Compiling serde_json v1.0.85
   Compiling futures-io v0.3.25
   Compiling rayon-core v1.9.3
   Compiling core-foundation-sys v0.8.3
   Compiling libz-ng-sys v1.1.8
   Compiling lz4-sys v1.9.4
   Compiling lexical-parse-integer v0.8.6
   Compiling lexical-write-integer v0.8.5
   Compiling alloc-no-stdlib v2.0.4
   Compiling async-trait v0.1.58
   Compiling zstd-safe v5.0.2+zstd.1.5.2
   Compiling snap v1.0.5
   Compiling lexical-parse-float v0.8.5
   Compiling lexical-write-float v0.8.5
   Compiling alloc-stdlib v0.2.2
   Compiling crossbeam-deque v0.8.2
   Compiling crossbeam-channel v0.5.6
   Compiling num_cpus v1.13.1
   Compiling rayon v1.5.3
   Compiling num-bigint v0.4.3
   Compiling ahash v0.8.1
   Compiling signal-hook v0.3.14
   Compiling parking_lot_core v0.9.4
   Compiling futures-macro v0.3.24
   Compiling fallible-streaming-iterator v0.1.9
   Compiling adler v1.0.2
   Compiling rle-decode-fast v1.0.3
   Compiling either v1.8.0
   Compiling libflate_lz77 v1.1.0
   Compiling miniz_oxide v0.5.4
   Compiling async-stream-impl v0.3.3
   Compiling lexical-core v0.8.5
   Compiling rustc_version v0.4.0
   Compiling brotli-decompressor v2.3.2
   Compiling iana-time-zone v0.1.53
   Compiling time v0.1.44
   Compiling signal-hook-registry v1.4.0
   Compiling num-iter v0.1.43
   Compiling num-rational v0.4.1
   Compiling rustversion v1.0.9
   Compiling crc-catalog v1.1.1
   Compiling array-init-cursor v0.2.0
   Compiling smallvec v1.9.0
   Compiling adler32 v1.2.0
   Compiling libflate v1.2.0
   Compiling brotli v3.3.4
   Compiling planus v0.3.1
   Compiling crc v2.1.0
   Compiling chrono v0.4.22
   Compiling arrow2 v0.15.0 (https://github.com/ritchie46/arrow2?branch=polars_2022-12-30#baa26189)
   Compiling futures-executor v0.3.24
   Compiling futures v0.3.24
   Compiling parquet-format-safe v0.2.4
   Compiling async-stream v0.3.3
   Compiling streaming-decompression v0.1.2
   Compiling multiversion-macros v0.6.1
   Compiling bytemuck_derive v1.2.1
   Compiling mio v0.8.5
   Compiling num-complex v0.4.2
   Compiling rand_core v0.6.4
   Compiling thiserror v1.0.37
   Compiling ppv-lite86 v0.2.16
   Compiling seq-macro v0.3.1
   Compiling simdutf8 v0.1.4
   Compiling avro-schema v0.3.0
   Compiling rand_chacha v0.3.1
   Compiling signal-hook-mio v0.2.3
   Compiling json-deserializer v0.4.4
   Compiling multiversion v0.6.1
   Compiling bytemuck v1.12.1
   Compiling arrow-format v0.8.1
   Compiling parking_lot v0.12.1
   Compiling thiserror-impl v1.0.37
   Compiling smartstring v1.0.1
   Compiling hash_hasher v2.0.3
   Compiling anyhow v1.0.65
   Compiling dyn-clone v1.0.9
   Compiling base64 v0.13.1
   Compiling ethnum v1.3.0
   Compiling rawpointer v0.2.1
   Compiling foreign_vec v0.1.0
   Compiling bitflags v1.3.2
   Compiling streaming-iterator v0.1.8
   Compiling heck v0.4.0
   Compiling strength_reduce v0.2.3
   Compiling crossterm v0.25.0
   Compiling strum_macros v0.24.3
   Compiling matrixmultiply v0.3.2
   Compiling num v0.4.0
   Compiling rand v0.8.5
   Compiling hashbrown v0.13.1
   Compiling aho-corasick v0.7.19
   Compiling unicode-width v0.1.10
   Compiling regex-syntax v0.6.27
   Compiling byteorder v1.4.3
   Compiling strum v0.24.1
   Compiling comfy-table v6.1.2
   Compiling fxhash v0.2.1
   Compiling rand_distr v0.4.3
   Compiling ndarray v0.15.6
   Compiling polars-utils v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling hex v0.4.3
   Compiling xxhash-rust v0.8.6
   Compiling halfbrown v0.1.15
   Compiling regex v1.6.0
   Compiling jsonpath_lib v0.3.0 (https://github.com/ritchie46/jsonpath?branch=improve_compiled#24eaf0b4)
   Compiling float-cmp v0.9.0
   Compiling value-trait v0.5.1
   Compiling now v0.1.3
   Compiling lexical v6.1.1
   Compiling dirs-sys v0.3.7
   Compiling libR-sys v0.3.0
   Compiling simd-json v0.7.0
   Compiling dirs v4.0.0
   Compiling memmap2 v0.5.7
   Compiling bytes v1.3.0
   Compiling libmimalloc-sys v0.1.28
   Compiling enum_dispatch v0.3.8
   Compiling extendr-engine v0.3.1 (https://github.com/extendr/extendr#43a086a2)
   Compiling pin-project-internal v1.0.12
   Compiling polars v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling extendr-api v0.3.1 (https://github.com/extendr/extendr#43a086a2)
   Compiling glob v0.3.0
   Compiling pin-project v1.0.12
   Compiling extendr-macros v0.3.1 (https://github.com/extendr/extendr#43a086a2)
   Compiling spin v0.9.4
   Compiling nanorand v0.7.0
   Compiling paste v1.0.7
   Compiling flate2 v1.0.24
   Compiling lazy_static v1.4.0
   Compiling flume v0.10.14
   Compiling state v0.5.3
   Compiling mimalloc v0.1.32
   Compiling lz4 v1.24.0
   Compiling zstd v0.11.2+zstd.1.5.2
   Compiling parquet2 v0.17.1
   Compiling polars-arrow v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling polars-core v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling polars-ops v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling polars-time v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling polars-io v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling polars-plan v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling polars-pipe v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling polars-lazy v0.26.1 (https://github.com/pola-rs/polars.git?rev=91a419acaf024e64410e77210ed928051aab76b5#91a419ac)
   Compiling rpolars v0.1.0 (/private/var/folders/sp/scbzkbwx6hbchmylsx0y52k80000gn/T/RtmpraKK28/R.INSTALL2fc76c285b86/rpolars/src/rust)
    Finished release [optimized] target(s) in 4m 59s
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib -L/opt/R/arm64/lib -o rpolars.so entrypoint.o -L./rust/target/release -lrpolars -F/Library/Frameworks/R.framework/Versions/4.2-arm64 -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: -undefined dynamic_lookup may not work with chained fixups
installing to /Users/dyfanjones/Library/R/arm64/4.2/library/00LOCK-rpolars/00new/rpolars/libs
** R
** byte-compile and prepare package for lazy loading

add syntax check to: DataFrame, DataType, DataTypeVector, Expr, LazyFrame, LazyGroupBy, ProtoExprArray, RNullValues, Series, VecDataFrame, 
[1] "Modifying extendr bindings, originals converted to pure functions and saved to rpolars:::.pr"

 setting public methods for  ^DataFrame_
 reuse internal method :
 get_columns, lazy
Internal methods not in use or replaced :
 by_agg, clone_see_me_macro, new, new_with_capacity, select_at_idx, set_column_from_robj, set_column_from_series, set_column_names_mut, to_list_tag_structs
 insert derived methods:
DataFrame_as_data_frame , DataFrame_clone , DataFrame_columns , DataFrame_dtypes , DataFrame_filter , DataFrame_get_column , DataFrame_groupby , DataFrame_height , DataFrame_join , DataFrame_limit , DataFrame_print , DataFrame_schema , DataFrame_select , DataFrame_shape , DataFrame_to_data_frame , DataFrame_to_list , DataFrame_to_series , DataFrame_to_struct , DataFrame_unnest , DataFrame_width , DataFrame_with_column , DataFrame_with_columns , 

 setting public methods for  ^LazyFrame_
 reuse internal method :
 describe_plan, filter, print, with_column
Internal methods not in use or replaced :

 insert derived methods:
LazyFrame_collect , LazyFrame_describe_optimized_plan , LazyFrame_groupby , LazyFrame_join , LazyFrame_limit , LazyFrame_select , LazyFrame_with_columns , 

 setting public methods for  ^Expr_
 reuse internal method :
 abs, add, agg_groups, alias, all, and, any, arccos, arccosh, arcsin, arcsinh, arctan, arctanh, arg_max, arg_min, arg_unique, ceil, cos, cosh, count, div, drop_nans, drop_nulls, eq, exp, explode, first, flatten, floor, gt, gt_eq, is_duplicated, is_finite, is_first, is_in, is_infinite, is_nan, is_not, is_not_nan, is_not_null, is_null, is_unique, keep_name, last, len, list, log10, lower_bound, lt, lt_eq, max, mean, median, min, mode, mul, n_unique, nan_max, nan_min, neq, null_count, or, product, rechunk, shift, shrink_dtype, sign, sin, sinh, sub, sum, tan, tanh, to_physical, unique_counts, upper_bound, xor
Internal methods not in use or replaced :
 col, cols, dtype_cols, exclude_dtype, fill_null_with_strategy, sample_frac, sample_n, unique_stable
 insert derived methods:
Expr_append , Expr_apply , Expr_arg_sort , Expr_argsort , Expr_backward_fill , Expr_cast , Expr_clip , Expr_clip_max , Expr_clip_min , Expr_cumcount , Expr_cummax , Expr_cummin , Expr_cumprod , Expr_cumsum , Expr_cumulative_eval , Expr_diff , Expr_dot , Expr_entropy , Expr_ewm_mean , Expr_ewm_std , Expr_ewm_var , Expr_exclude , Expr_extend_constant , Expr_extend_expr , Expr_fill_nan , Expr_fill_null , Expr_filter , Expr_forward_fill , Expr_hash , Expr_head , Expr_inspect , Expr_interpolate , Expr_is_between , Expr_kurtosis , Expr_limit , Expr_lit , Expr_log , Expr_map , Expr_map_alias , Expr_over , Expr_pct_change , Expr_pow , Expr_prefix , Expr_print , Expr_quantile , Expr_rank , Expr_reinterpret , Expr_rep , Expr_rep_extend , Expr_repeat_by , Expr_reshape , Expr_reverse , Expr_rolling_max , Expr_rolling_mean , Expr_rolling_median , Expr_rolling_min , Expr_rolling_quantile , Expr_rolling_skew , Expr_rolling_std , Expr_rolling_sum , Expr_rolling_var , Expr_round , Expr_rpow , Expr_sample , Expr_search_sorted , Expr_set_sorted , Expr_shift_and_fill , Expr_shuffle , Expr_skew , Expr_slice , Expr_sort , Expr_sort_by , Expr_sqrt , Expr_std , Expr_suffix , Expr_tail , Expr_take , Expr_take_every , Expr_to_r , Expr_top_k , Expr_unique , Expr_value_counts , Expr_var , Expr_where , 

 setting public methods for  ^Series_
 reuse internal method :
 alias, any, arg_max, arg_min, chunk_lengths, clone, len, max, min, sum, to_frame
Internal methods not in use or replaced :
 append_mut, is_sorted_flag, is_sorted_reverse_flag, new, panic, rename_mut, set_sorted_mut, sleep, sort_mut
 insert derived methods:
Series_abs , Series_add , Series_all , Series_append , Series_apply , Series_ceil , Series_compare , Series_cumsum , Series_div , Series_dtype , Series_flags , Series_floor , Series_is_numeric , Series_is_unique , Series_mul , Series_name , Series_print , Series_rem , Series_rename , Series_rep , Series_series_equal , Series_set_sorted , Series_shape , Series_sort , Series_std , Series_sub , Series_to_r , Series_to_r_list , Series_to_r_vector , Series_value_counts , Series_var , [1] ""
[1] "done source"
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (rpolars)
sorhawell commented 1 year ago

@DyfanJones nice that is very helpful, and a very clean install script I think 💯

JosiahParry commented 1 year ago

I'll give these a shot this weekend! Thank you both :)

sorhawell commented 1 year ago

Hey YOU! Are you're looking for a an easy pre-compiled rpolars build for Arch64/Arm64 Darwin?

I'd like to support that, and think it is quite possible with a little cross-compilation. All I need is someone with a Arm64 machine to unittest the builds. Just comment with a "Yeah" and I will provide a beta binary install within a week or so.

DyfanJones commented 1 year ago

@sorhawell happy to help. let me know what is required

sorhawell commented 1 year ago

@DyfanJones uhh excellent :)

Are you using a Arm64 macOS (aarch64-apple-darwin) or perhaps a Arm64 Linux machine? (aarch64-unknown-linux-gnu)

I will add some install.package one-liners in the readme to try out. I write when they are ready.

list of other rust platforms, preferably gnu or darwin

DyfanJones commented 1 year ago

I have Arm64 macOS. Currently I have the following rust setup so I can currently install rpolars. However I am happy to remove this for any testing that is required.

# terminal setup
# https://rustup.rs/
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# used default option 1

rustup toolchain install nightly
rustup target add aarch64-apple-darwin --toolchain nightly
rustup default nightly-aarch64-apple-darwin
sorhawell commented 1 year ago

@DyfanJones Is it by chance possible for you to install via R-universe like this? 

 options(repos = c(
   rpolars = 'https://rpolars.r-universe.dev',
   CRAN = 'https://cloud.r-project.org')
 )
# Download and install rpolars in R
 install.packages('rpolars')

R-universe is very alpha and only 40% of workflows passes so far

DyfanJones commented 1 year ago

I was able to install rpolars from r-universe however I had to install from source:

options(repos = c(
  rpolars = 'https://rpolars.r-universe.dev',
  CRAN = 'https://cloud.r-project.org')
)
# Download and install rpolars in R
install.packages('rpolars')

Installing package into ‘/Users/dyfanjones/Library/R/arm64/4.2/library’
(as ‘lib’ is unspecified)
Package which is only available in source form, and may need compilation of C/C++/Fortran: ‘rpolars’
JosiahParry commented 1 year ago

I was able to install from r-universe. It required me to compile from source which took some time but it worked. This is better than me not being able to install at all from GitHub or a clone

sorhawell commented 1 year ago

@DyfanJones How about does this work for you? It will download a ~200Mb precompiled object file and let your R+Xcode link and build. It should take around 30-60sec to build the package then.

#install M1 arch from cross-compiled binary
remotes::install_github("https://github.com/pola-rs/r-polars",ref = "long_arms64", force =TRUE)

for me it looks like this (NB I set an extra variable env_arch="x86_64" to not download M1/arm64 object file). If env_arch is not set or set to anything else it chooses M1/arm64

> remotes::install_github("https://github.com/pola-rs/r-polars",ref = "long_arms64")
Downloading GitHub repo pola-rs/r-polars@long_arms64
── R CMD build ───────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/v1/b2c26lpn2yjd997jg_gn4fgc0000gn/T/RtmpibzVGk/remotes3867444c33d9/pola-rs-r-polars-e8d3c7f/DESCRIPTION’ ...
─  preparing ‘rpolars’:
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts (476ms)
─  checking for empty or unneeded directories
─  building ‘rpolars_0.4.4.tar.gz’

Installing package into ‘/Users/sorenwelling/Library/Caches/org.R-project.R/R/renv/library/r-polars-82a2433f/R-4.2/x86_64-apple-darwin17.0’
(as ‘lib’ is unspecified)
* installing *source* package ‘rpolars’ ...
** using staged installation
** libs
rm -Rf rpolars.so ./rust/target/release/librpolars.a entrypoint.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c entrypoint.c -o entrypoint.o
echo x86_64
x86_64
# arch cross compile arch
if [ "x86_64" == "x86_64" ]; \
    then \
        rpolars_url_binary="https://github.com/pola-rs/r-polars/releases/download/m1test12/rpolars_binary_x86_64-apple-darwin.a"; \
    else \
        rpolars_url_binary="https://github.com/pola-rs/r-polars/releases/download/m1test12/rpolars_binary_aarch64-apple-darwin.a"; \
    fi; \
    echo $rpolars_url_binary; \
    if [ "true"  == "true" ]; \
    then \
            echo "fetch!!"; \
        mkdir -p ./rust/target/release; \
        curl -L -o ./rust/target/release/librpolars.a $rpolars_url_binary; \
  else \
        curdir=/private/var/folders/v1/b2c26lpn2yjd997jg_gn4fgc0000gn/T/RtmpzLvnC6/R.INSTALL8bfd6272ae50/rpolars/src; \
        cd ./rust/..; \
        cargo build --lib --release --manifest-path="./rust/Cargo.toml"; \
        cd ; \
    fi
https://github.com/pola-rs/r-polars/releases/download/m1test12/rpolars_binary_x86_64-apple-darwin.a
fetch!!
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  186M  100  186M    0     0  7905k      0  0:00:24  0:00:24 --:--:-- 8147k
# if compilation did take place "here" but in RPOLARS_RUST_SOURCE then symlink the binary to here
if [ -f "./rust/target/release/librpolars.a" ]; \
    then \
    echo "file is there: "; \
    else \
    echo "no, file is NOT there: "; \
    mkdir -p ./rust/target/release ; \
    echo "trying to symlink in "./rust/target/release/librpolars.a""; \
    ln -s "./rust/target/release/librpolars.a" ./rust/target/release/librpolars.a ; \
    fi
file is there: 
# clean up any kind of files R CMD check does not like
if [ "nope" == "true" ]; \
    then \
    echo "cleanup!!" ; \
    mv ./rust/target/release/librpolars.a ./rust/target/release/../temp_binary.a; \
    rm -rf ./rust/target/release; \
    mkdir ./rust/target/release; \
    mv ./rust/target/release/../temp_binary.a ./rust/target/release/librpolars.a; \
    rm -rf ./src/.cargo; \
    else \
    echo "hands off!!" ; \
    fi
hands off!!
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/opt/gettext/lib -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib -o rpolars.so entrypoint.o -L./rust/target/release -lrpolars -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (rpolars)
DyfanJones commented 1 year ago

Hi @sorhawell, this works for me :) My internet is a little slower than yours and took around 30-40 seconds, however it installed 😄

sorhawell commented 1 year ago

@DyfanJones many thanks for fast test :)

I may potentially use a compressed object file in a later iteration, of size 20-40 Mb. But then I have to make sure every machine has the same compression tool available ... hmm something tar.gz probably is

I will polish this a bit more and should become a part of the CI/CD cycle in a week or so.

eitsupi commented 1 year ago

FYI, to support arm64 Linux, we need to bump the version of extendr and use libR-sys 0.4.0. https://github.com/extendr/libR-sys/releases/tag/v0.4.0

sorhawell commented 1 year ago

bumping extendr_api to 0.4 #66

eitsupi commented 1 year ago

Currently, the latest polars are automatically deployed in R-universe (it may take up to 3 hours to deploy). So installing from there should work in most cases.

If you encounter any problems, please feel free to reopen it.

sorhawell commented 1 year ago

@DyfanJones @JosiahParry @coforfe

I have made some progress on cross-compilation, this one still requires make + Xcode (or another newer compiler) in path, but no rust toolchain is needed. Does this work for you? The install should take ~30 sec.

for mac m2 and alike

install.packages("https://github.com/pola-rs/r-polars/releases/download/cross_test9/polars_cross_aarch64-apple-darwin.tar.gz",repos = NULL,type = "source")

for mac intel this works for me

install.packages("https://github.com/pola-rs/r-polars/releases/download/cross_test9/polars_cross_x86_64-apple-darwin.tar.gz",repos = NULL,type = "source")
DyfanJones commented 1 year ago

@sorhawell This works for me.