Git submodules are not found when using argument `subdir`

[david-cortes]

If one tries to install a package from github that has git submodules, and the installation is not from the root of the repository, the submodules will get cloned during the install, but somehow are not picked by the installer.

Example:

remotes::install_github("dmlc/xgboost", subdir="R-package")

Downloading GitHub repo dmlc/xgboost@HEAD
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules --branch main https://github.com/dmlc/dmlc-core /tmp/RtmpqeXs2V/remotes3ee9693120d7/dmlc-xgboost-b807f3e/R-package/../dmlc-core
Cloning into '/tmp/RtmpqeXs2V/remotes3ee9693120d7/dmlc-xgboost-b807f3e/R-package/../dmlc-core'...
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/rapidsai/gputreeshap.git /tmp/RtmpqeXs2V/remotes3ee9693120d7/dmlc-xgboost-b807f3e/R-package/../gputreeshap
Cloning into '/tmp/RtmpqeXs2V/remotes3ee9693120d7/dmlc-xgboost-b807f3e/R-package/../gputreeshap'...

...

make: *** No rule to make target '../..//src/metric/metric.o', needed by 'xgboost.so'.  Stop.
make: *** Waiting for unfinished jobs....
xgboost_custom.cc:5:10: fatal error: xgboost/logging.h: No such file or directory
    5 | #include <xgboost/logging.h>
      |          ^~~~~~~~~~~~~~~~~~~
xgboost_R.cc:4:10: fatal error: dmlc/common.h: No such file or directory
    4 | #include <dmlc/common.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
compilation terminated.

One can verify that installation would succeed if doing the cloning and installing without remotes:

git clone --recursive https://github.com/dmlc/xgboost.git
cd xgboost/R-package
R CMD INSTALL .

(this is assuming that the R environment used there already has the xgboost dependencies installed)

[gaborcsardi]

As we focus on https://gihub.com/r-lib/pak now, it is unlikely that this will be implemented here.