Install from subdir with dependencies in other subdir

[schlegelp]

Hi.

I have a repository containing a Rust crate and separate R (using extendr) and Python bindings for it.

The project's layout is this:

.
├── fastcore           # the Rust crate
├── py/navis-fastcore  # Python bindings for `fastcore`
└── R/nat.fastcore     # R bindings for `fastcore`

The project uses the concept of Rust "workspaces" so that the Rust code in R/nat.fastcore and py/navis-fastcore knows to look for the fastcore dependency in the parent directory.

The whole thing compiles just fine locally via rextendr::document. For user installs, the folks from @extendr (tagging in case they like to chime in) suggest using remotes::install_github which unfortunately fails:

> remotes::install_github("schlegelp/fastcore-rs", subdir="R/nat.fastcore")
Downloading GitHub repo schlegelp/fastcore-rs@HEAD
── R CMD build ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   checking for file ‘/private/var/folders/d1/ql9nlj6j4gndbkzbgrwbfkq80000gn/T/RtmpfGM9jr/remotes12a556a6e71cd/schlegelp-fastcore-rs-fec485f/R/nat.fastcore/✔  checking for file ‘/private/var/folders/d1/ql9nlj6j4gndbkzbgrwbfkq80000gn/T/RtmpfGM9jr/remotes12a556a6e71cd/schlegelp-fastcore-rs-fec485f/R/nat.fastcore/DESCRIPTION’
─  preparing ‘nat.fastcore’:
✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘nat.fastcore_0.0.0.9000.tar.gz’

* installing *source* package ‘nat.fastcore’ ...
** using staged installation
** libs
using C compiler: ‘Apple clang version 15.0.0 (clang-1500.3.9.4)’
using SDK: ‘MacOSX14.4.sdk’
rm -Rf nat.fastcore.so ./rust/target/release/libnat_fastcore.a entrypoint.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c entrypoint.c -o entrypoint.o
# In some environments, ~/.cargo/bin might not be included in PATH, so we need
# to set it here to ensure cargo can be invoked. It is appended to PATH and
# therefore is only used if cargo is absent from the user's PATH.
if [ "" != "true" ]; then \
        export CARGO_HOME=/private/var/folders/d1/ql9nlj6j4gndbkzbgrwbfkq80000gn/T/RtmpR18qIM/R.INSTALL16b8154334f73/nat.fastcore/src/.cargo; \
    fi && \
        export PATH="/Users/philipps/projects/elastix-5/bin:/Users/philipps/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/X11/bin:/Users/philipps/.cargo/bin:/Users/philipps/.cargo/bin" && \
        cargo build --lib --release --manifest-path=./rust/Cargo.toml --target-dir ./rust/target
error: failed to parse manifest at `/private/var/folders/d1/ql9nlj6j4gndbkzbgrwbfkq80000gn/T/RtmpR18qIM/R.INSTALL16b8154334f73/nat.fastcore/src/rust/Cargo.toml`

Caused by:
  error inheriting `fastcore` from workspace root manifest's `workspace.dependencies.fastcore`

Caused by:
  failed to find a workspace root
make: *** [rust/target/release/libnat_fastcore.a] Error 101
ERROR: compilation failed for package ‘nat.fastcore’
* removing ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/nat.fastcore’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/d1/ql9nlj6j4gndbkzbgrwbfkq80000gn/T//RtmpfGM9jr/file12a551c528764/nat.fastcore_0.0.0.9000.tar.gz’ had non-zero exit status

The error message suggests that Rust's cargo build can't find the root folder with the fastcore Rust crate.

Correct me if I'm wrong but it looks like install_remotes first clones the whole repository (I can still find remnants of it in a temporary folder) but then moves only the subdir somewhere else before building the R library, at which point the fastcore crate is not where it's expected anymore.

I appreciate the setup is rather complicated and I'm half expecting this to simply not be possible with install_github but I thought it's worth asking if you could think of a fix/workaround.

(CC @jefferis)