Closed joelnitta closed 6 months ago
I don't know if switching R version is the problem here. Does it work with R 4.2? I.e. maybe the binary for {assertr} for R 4.2 is not compatible with R 4.3 and renv can
Can you try to install the required version of assertr with {renv} in a fresh renv and see if that works? And then retry pre-commit? Maybe once it's cached, it works.
- I don't know if switching R version is the problem here. Does it work with R 4.2? I.e. maybe the binary for {assertr} for R 4.2 is not compatible with R 4.3
I tried with R 4.2 and precommit still fails. However, when I try install.packages("assertr")
in R 4.2.3, I get an error:
tcltk DLL is linked to '/usr/local/lib/libtk8.6.dylib'
Error: .onLoad failed in loadNamespace() for 'tcltk', details:
call: fun(libname, pkgname)
error: Tcl/Tk libraries are missing: install the Tcl/Tk component from the R installer
But I don't see this error in v4.3.0 - {assertr} can be installed successfully. So I think it is as you say: the binary for {assertr} for R 4.2 is not compatible with R 4.3.
- Can you try to install the required version of assertr with {renv} in a fresh renv and see if that works? And then retry pre-commit? Maybe once it's cached, it works.
I'm unclear on what renv.lock
file {precommit} is using. I thought it always uses https://github.com/lorenzwalthert/precommit/blob/main/renv.lock, so there is no point in me trying to create my own renv.lock?
If indeed the problem is that the current renv.lock
in {precommit} is no longer compatible with R 4.3.0 for some packages, could it be updated?
With additional_dependencies
, pre-commit just installs into the renv that you have, without updating any lockfile. So maybe you can clone the {precommit} package, run renv::restore()
, and then renv::install('yourpkg@version')
and see if that also fails? If it does, maybe renv::purge()
the package and retry?
If this already fails at restore()
, we know the problem is not related to {assertr}.
Thanks for the explanation.
After cloning {precommit}, I am able to run renv::restore()
then renv::install("assertr@3.0.0")
.
I did see a warning that the lockfile was out of date before restoring, but ignored it.
FYI I frequently encounter warnings like this:
> renv::snapshot()
WARNING: One or more problems were discovered while enumerating dependencies.
/Users/joelnitta/repos/precommit/tests/testthat/in/parsable-R-fail.R
--------------------------------------------------------------------
ERROR 1: /Users/joelnitta/repos/precommit/tests/testthat/in/parsable-R-fail.R:1:3: unexpected numeric constant
1: 1 1
^
/Users/joelnitta/repos/precommit/tests/testthat/in/parsable-R-fail.Rmd
----------------------------------------------------------------------
ERROR 1: <text>:7:3: unexpected numeric constant
6:
7: 1 1
^
/Users/joelnitta/repos/precommit/tests/testthat/in/parsable-R-success.Rmd
-------------------------------------------------------------------------
ERROR 1: <text>:16:11: unexpected numeric constant
15:
16: fas / fk 12
^
/Users/joelnitta/repos/precommit/tests/testthat/in/style-files-fail-parse.R
---------------------------------------------------------------------------
ERROR 1: /Users/joelnitta/repos/precommit/tests/testthat/in/style-files-fail-parse.R:1:4: unexpected ')'
1: 1+ )
^
Please see `?renv::dependencies` for more information.
Do you want to proceed? [y/N]: y
Ok. Yes, these warnings were expected. Some test files are not parsable on purpose (to test functions). I suggest you pre-commit clean
in the repo where you want to use the hooks and purge the assertr package with {renv} from anywhere and try again to run pre-commit.
Sorry, can you clarify what you mean by "purge the assertr package with {renv} from anywhere"?
The package where I want to use precommit does not have a renv.lock file (does not use renv) so I can't "purge" it.
renv::purge("assertr@3.0.0")
seems a global operation independent of an activated renv, so can you run this from any directory? Otherwise from where you have installed the package in (cloned {precommit}).
> renv::purge("assertr@3.0.0")
* The requested package is not installed in the cache -- nothing to do.
I have the same behavior with 4.3.0
$ pre-commit
[INFO] Initializing environment for https://github.com/compilerla/conventional-pre-commit.
[INFO] Initializing environment for https://github.com/lorenzwalthert/precommit.
[INFO] Initializing environment for https://github.com/lorenzwalthert/precommit:ggplot2,scales,sf,dplyr,lwgeom,cowplot,purrr,git2r.
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/pre-commit-ci/pre-commit-ci-config.
[INFO] Installing environment for https://github.com/lorenzwalthert/precommit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/lorenzwalthert/precommit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/Library/Frameworks/R.framework/Resources/bin/Rscript', '--no-save', '--no-restore', '--no-site-file', '--no-environ', '-e', ' options(install.packages.compile.from.source = "never", pkgType = "binary")\n renv::install(commandArgs(trailingOnly = TRUE))\n ', 'ggplot2', 'scales', 'sf', 'dplyr', 'lwgeom', 'cowplot', 'purrr', 'git2r')
return code: 1
stdout:
ℹ Using R 4.3.0 (lockfile was generated with R 4.2.1)
ℹ Using R 4.3.0 (lockfile was generated with R 4.2.1)
stderr:
Error: package 'ggplot2' is not available
In addition: Warning message:
no MRAN records available from repository URL '/bin/macosx/big-sur-x86_64/contrib/4.3'
Traceback (most recent calls last):
7: renv::install(commandArgs(trailingOnly = TRUE))
6: retrieve(names(remotes))
5: handler(package, renv_retrieve_impl(package))
4: renv_retrieve_impl(package)
3: renv_available_packages_latest(package)
2: stopf("package '%s' is not available", package)
1: stop(sprintf(fmt, ...), call. = call.)
Execution halted
Check the log at /Users/maikol/.cache/pre-commit/pre-commit.log
This is the log file.
The problem in my config is this chunk, generated by precommit::snippet_generate('additional-deps-roxygenize')
.
- id: roxygenize
# roxygen requires loading pkg -> add dependencies from DESCRIPTION
additional_dependencies:
- ggplot2
- scales
- sf
- dplyr
- lwgeom
- cowplot
- purrr
I removed it, and precommit is working again.
I can confirm that removing the id: roxygenize
chunk allows precommit to work, and putting it back results in the error again.
@lorenzwalthert do you mind explaining why you need to install all of the package dependencies in the precommit renv environment?
they need to be installed because running roxygen requires pkgload::load_all()
as mentioned in #243 and https://github.com/r-lib/roxygen2/issues/771. Hence, we'll probably create a new hook with language: script
in the future to rely on the global R library for roxygenise.
no MRAN records available from repository URL '/bin/macosx/big-sur-x86_64/contrib/4.3'
I don't know if the problem is alwo that MRAN has no binaries built for 4.3 and since it will retire soon, maybe they don't plan to do so in the future at all. May be an issue for {renv} in general, see https://github.com/rstudio/renv/issues/1179.
So I guess a reasonable work-around for now is to turn off the roxygenize
pre-commit check (so I won't be checking for .Rd files rendered after changes to the source). Better than running git commit --no-verify
everytime I suppose.
My above suggestions:
pre-commit clean
to delete pre-commit cache.attempt to commit. If does not work, yes, then you can de-activate the hook by commenting the id of the hook out in .pre-commit-config.yaml
.
I appear to be having a similar issue, albeit not related to the package assertr
, but other packages required in the installation of the pre-commit renv. The issue started after I upgraded from R 4.2.1 to 4.3.
I've attached the pre-commit log below:
The precommit yaml I am using is:
# All available hooks: https://pre-commit.com/hooks.html
# R specific hooks: https://github.com/lorenzwalthert/precommit
repos:
- repo: https://github.com/lorenzwalthert/precommit
rev: v0.3.2.9007
hooks:
- id: style-files
args: [--style_pkg=styler, --style_fun=tidyverse_style]
- id: spell-check
exclude: >
(?x)^(
.*\.[rR]|
.*\.feather|
.*\.jpeg|
.*\.pdf|
.*\.png|
.*\.py|
.*\.RData|
.*\.rds|
.*\.Rds|
.*\.Rproj|
.*\.sh|
(.*/|)\.gitignore|
(.*/|)\.gitlab-ci\.yml|
(.*/|)\.lintr|
(.*/|)\.pre-commit-.*|
(.*/|)\.Rbuildignore|
(.*/|)\.Renviron|
(.*/|)\.Rprofile|
(.*/|)\.travis\.yml|
(.*/|)appveyor\.yml|
(.*/|)NAMESPACE|
(.*/|)renv/settings\.dcf|
(.*/|)renv\.lock|
(.*/|)WORDLIST|
\.github/workflows/.*|
data/.*|
)$
- id: lintr
- id: readme-rmd-rendered
- id: parsable-R
- id: no-browser-statement
- id: no-debug-statement
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-added-large-files
args: ['--maxkb=200']
- id: end-of-file-fixer
exclude: '\.Rd'
- repo: https://github.com/pre-commit-ci/pre-commit-ci-config
rev: v1.5.1
hooks:
# Only required when https://pre-commit.ci is used for config validation
- id: check-pre-commit-ci-config
- repo: local
hooks:
- id: forbid-to-commit
name: Don't commit common R artifacts
entry: Cannot commit .Rhistory, .RData, .Rds or .rds.
language: fail
files: '\.(Rhistory|RData|Rds|rds)$'
# `exclude: <regex>` to allow committing specific files
ci:
autoupdate_schedule: monthly
To reproduce
precommit::use_precommit()
pre-commit
on command line within project directoryAdditional context
precommit
} with remotes::install_github('lorenzwalthert/precommit')
brew update pre-commit
renv
}: NopackageVersion('renv')
: 0.17.3@RHReynolds can you folllow the instructions I put in the Wiki to deal with these kinds of errors and tell me if this helped to resolve the problem? Another option is that maybe the combination
options(install.packages.compile.from.source = "never", pkgType = "binary")
is not working well with package manager, as the URL that is produced seems invalid:
Warning: failed to retrieve 'https://packagemanager.rstudio.com/all/latest/src/contrib/R.cache_0.16.0.tgz' [error code 22]
Maybe I'll file an issue in {renv} to see if the source is there. In the meantime, you can also update to {renv} 1.0.0.
Also note that it seems like binary packages are now provided on the public Posit Package Manager also for macOS, so all major platfroms are supported. This might alleviate or solve the problem.
I'm not sure I understand the issue completely, but right now I'm having the following problem (which looks related to the ones above) when running the roxygenize
hook:
roxygenize...............................................................Failed
- hook id: roxygenize
- exit code: 1
ℹ Using R 4.3.1 (lockfile was generated with R 4.2.1)
ℹ Using R 4.3.1 (lockfile was generated with R 4.2.1)
ℹ Loading kofadapters
Error in `precommit::roxygenize_with_cache()`:
! The package `macrobondr` is required.
Backtrace:
▆
1. └─precommit::roxygenize_with_cache(key = wd, dirs = path_relative_cache)
2. └─rlang::abort(conditionMessage(out))
Execution halted
If I understood the comments correctly, adding macrobondr
to my .pre-commit-config.yaml
in the additional_dependencies
key for the roxygenize
hook should fix it.
I see 2 big issues that prevents me from using this hook in any of the packages I work with, which is a shame:
DESCRIPTION
and once in the pre-commit configuration fileIs this really how the hook works or am I missing something else?
I would have to repeat my dependencies twice: once in DESCRIPTION and once in the pre-commit configuration file
yes, for the roxygenise hook as it is implemented now, this is the way to go. We might create a new hook in the future that relies on your global R library to avoid this (https://github.com/lorenzwalthert/precommit/issues/494).
Since this package is only available through a private CRAN, the installation fails anyway
Your package must be installable through renv::install()
, which supports also packages hosted on GitHub or Bioconductor. I don't know how you configure {renv} to work with private repositories, but if it's through env variables that are set before R starts up (note that R pre-commit hooks emulate --vanilla
and .Rprofile
and .Renviron
are not considered), it should work.
yes, for the roxygenise hook as it is implemented now, this is the way to go. We might create a new hook in the future that relies on your global R library to avoid this (#494).
I use renv, so my R library isn't global, but local to the package. Which also means that the library loaded by roxygenize
might be out-of-sync with my own renv.lock
, so different versions. It might not change much in practice, but still there is a mismatch.
Your package must be installable through
renv::install()
, which supports also packages hosted on GitHub or Bioconductor. I don't know how you configure {renv} to work with private repositories, but if it's through env variables that are set before R starts up (note that R pre-commit hooks emulate--vanilla
and.Rprofile
and.Renviron
are not considered), it should work.
How do you setup anything before startup if .Rprofile
and .Renviron
are not considered?
The normal way to add a CRAN repository is by options(repos = c(CRAN = "https://....", PRIVATECRAN="https://cran.vpnonly.mycompany.com"))
, which is both in my .Rprofile
for the initial renv::init()
, and it is saved in the renv.lock
In fact the hook could easily detect the presence of an renv.lock
file and use that to restore the necessary packages - which would be lightning fast since they are already in the renv cache.
I use renv, so my R library isn't global, but local to the package. Which also means that the library loaded by roxygenize might be out-of-sync with my own renv.lock, so different versions. It might not change much in practice, but still there is a mismatch.
Yes, in that case, renv should be used by that hook.
How do you setup anything before startup if .Rprofile and .Renviron are not considered? The normal way to add a CRAN repository is by options(repos = c(CRAN = "https://....", PRIVATECRAN="https://cran.vpnonly.mycompany.com")), which is both in my .Rprofile for the initial renv::init(), and it is saved in the renv.lock
renv.lock
is given precedence over getOptions('repos')
with renv::install()
, so the repo name will already be right. My question was rather if you need to set some other auth before installation.
If you already have these packages installed and they are in your renv cache, then yes, that cache would be used and auth would not be a problem. You can even go so far and say fine, if the packages can't be installed, then you just have to install them manually interactively first and then rely on the cache.
renv.lock
is given precedence overgetOptions('repos')
withrenv::install()
, so the repo name will already be right. My question was rather if you need to set some other auth before installation.
There is some confusion here because there are two different renv.lock
files at play here: the one in my package which I normally use (and has the dependencies), and the one in the pre-commit
cache used by the hook (which only has the packages specified in additional_dependencies
). The one used by the hook does not have the additional repositories that I configured and thus cannot install private packages.
The private repositories in this case have no authentication but are only accessible from within certain networks.
If you already have these packages installed and they are in your renv cache, then yes, that cache would be used and auth would not be a problem. You can even go so far and say fine, if the packages can't be installed, then you just have to install them manually interactively first and then rely on the cache.
The hooks also run on the CI (in case someone doesn't install pre-commit
or take the --no-verify
shortcut), so manual installations won't suffice
There is some confusion here because there are two different renv.lock files at play here: the one in my package which I normally use (and has the dependencies), and the one in the pre-commit cache used by the hook (which only has the packages specified in additional_dependencies). The one used by the hook does not have the additional repositories that I configured and thus cannot install private packages.
Ok, yes, you are right, my bad.
The hooks also run on the CI (in case someone doesn't install pre-commit or take the --no-verify shortcut), so manual installations won't suffice
Ok, then this is not an option yes.
If you want to contribute to #494, you are most welcome. It would actually be mostly identical to the existing hook, maybe we can factor the script at inst/hook/exported/...
into a function that can then be called from both hooks. Or maybe we don't even need two hook scripts and we can simply change language
in .pre-commit-hooks.yaml
and check if a local renv should override the active library (either global or pre-commit's renv). I unfortunately don't have a lot of time for that kind of work currently. But I am happy to give guidance.
I have a solution for anyone who experiences issues with additional_dependencies
AND installed R via homebrew on their MacOS. Replace homebrew R installation with a manual installation (or one via rig
).
I noticed that additional_dependencies
would fail with a 404 from CRAN for some reason. Long story short, I arrived at a conclusion that .Platform$pkgType
and getOption("pkgType")
are both "source"
for my homebrew R installation. On the other hand, pre-commit enforces a "binary"
package type. I believe that mix of these two makes us end up with a malformed URL.
I asked my teammates to check their package type settings, and I have noticed a pattern where a homebrew R installation had these values set to "source"
, whereas a manual R installation or one via rig
, had .Platform$pkgType
set to "mac.binary.big-sur-arm64"
(on MacBook M-series) and getOption("pkgType")
set to "both"
. Once I switched to R installed via rig
everything started to work. I have settled for this solution and did not dive into why there is such a difference (a wild guess: brew compiles R and is missing some flags).
I hope this helps someone.
@TymekDev thanks for the hints. You are making the right connections. The goal was to enable fast binary installs on all platforms, so we tickered with these options as by default, it seemed as sometimes source installations were attempted. Some of these failed because the required built-time dependencies were missing on local computers (e.g. make). I have eliminated one dependency that would be problematic in this repo with #497. So maybe, we should not require binaries anymore.
Unfortunately, to me, it's still a mistry how install.packages.compile.from.source = "never"
and pkgType = "binary"
play together, let alone .Platform
and getOption("pkgType")
.
On the other hand, pre-commit enforces a "binary" package type. I believe that mix of these two makes us end up with a malformed URL.
Can you pin it down if it's a {renv} issue or a base R issue? What does the URL look like?
renv::restore()
relies on renvl.lock
and we specify posit package manager there, which supports binaries for all major platforms, so I am not sure how any of the other options play in here...
Hi @lorenzwalthert I'm having the exact same problem as mentioned before, on a Apple M1 Pro. I am having a problem with the roxygenize
hook with rstan
specifically, and it seems this problem has not been resolved yet. I followed the suggestion by @TymekDev to install R using rig
instead, but without success. Any suggestions on how to get this fixed?
Content of .pre-commit-config.yml
:
# All available hooks: https://pre-commit.com/hooks.html
# R specific hooks: https://github.com/lorenzwalthert/precommit
repos:
- repo: https://github.com/lorenzwalthert/precommit
rev: v0.4.0
hooks:
- id: roxygenize
# roxygen requires loading pkg -> add dependencies from DESCRIPTION
additional_dependencies:
- rstan
- R6
- rstudioapi
- stringr
- id: use-tidy-description
Output of running the precommit hook:
$ pre-commit run roxygenize
[INFO] Initializing environment for https://github.com/lorenzwalthert/precommit.
[INFO] Initializing environment for https://github.com/lorenzwalthert/precommit:rstan,R6,rstudioapi,stringr.
[INFO] Installing environment for https://github.com/lorenzwalthert/precommit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/usr/local/bin/Rscript', '--no-save', '--no-restore', '--no-site-file', '--no-environ', '/var/folders/ql/0csjgttj5vjdclw8jlpb74sw0000gq/T/tmplz1q5dnu/script.R', 'rstan', 'R6', 'rstudioapi', 'stringr')
return code: 1
stdout:
Retrieving 'https://mran.microsoft.com/snapshot/2023-12-06/bin/macosx/big-sur-arm64/contrib/4.3/rstan_2.32.3.tgz' ...
ERROR [curl: (6) Could not resolve host: mran.microsoft.comcurl: (6) Could not resolve host: mran.microsoft.comcurl: (6) Could not resolve host: mran.microsoft.comcurl: (6) Could not resolve host: mran.microsoft.com]
stderr:
Error: error downloading 'https://mran.microsoft.com/snapshot/2023-12-06/bin/macosx/big-sur-arm64/contrib/4.3/rstan_2.32.3.tgz' [curl: (6) Could not resolve host: mran.microsoft.comcurl: (6) Could not resolve host: mran.microsoft.comcurl: (6) Could not resolve host: mran.microsoft.comcurl: (6) Could not resolve host: mran.microsoft.com]
Traceback (most recent calls last):
8: renv::install(commandArgs(trailingOnly = TRUE))
7: retrieve(names(remotes))
6: handler(package, renv_retrieve_impl(package))
5: renv_retrieve_impl(package)
4: renv_retrieve_repos(record)
3: renv_retrieve_repos_impl(record)
2: renv_retrieve_package(record, url, path)
1: stop(status)
Execution halted
Check the log at /Users/kasparhageman/.cache/pre-commit/pre-commit.log
I'm not sure why renv
decides to use the MRAN repository, instead of the packagemanager.posit.co/
as declared in the renv.lock
file of this precommit
repository.
In any case, a quick fix for now has been to fork this repo (here) change the repository URL in the renv.lock
file, and point the pre-commit configuration of my project to use the forked repository instead, and this make setting up the precommit environment work.
It seems like rstan is not available as a binary from PPM for macOS according to https://packagemanager.posit.co/client/#/repos/cran/packages/rstan/631497/overview?search=rstan%23package-details
Hence, renv seems to fallback on MRAN, which does not have the package either.
I think the workaround would be to install the package outside of pre-commit (actually anywhere, you can try in your R console with renv::install()
) and count on {renv}'s global package cache (make sure you use the right version of the package).
To ensure that actually works, can you first renv::purge('rstan')
to remove the package, change back to this repo as the hook repo instead of your fork, then confirm you can reproduce the intial error and then try the manual installation?
Thanks for reaching out so quick.
After uninstalling and purging rstan from R and changing back to the repo, I do indeed get the same error as before.
After some debugging, I figured out that the version of rstan
(and StanHeaders
) installed by renv::install
and the precommit installation. So I needed to install the matching version through renv
to have precommit fetch the correct repository:
renv::install('rstan@2.32.3')
renv::install("StanHeaders@2.26.28")
The newest version of rstan
installed by renv
by default is 2.36.6
, hence the mismatch. Problem solved, thanks a lot!
Great. You can also pin the version in additional_dependencies
in your .pre-commit-config.yaml
if that was not clear before.
Let's close this. Anyone who finds new problems please open a new issue, it's already pretty convoluted.
Before filing a bug
remotes::install_github('lorenzwalthert/precommit')
precommit::autoupdate()
)I am getting the following error when I try to make a commit using precommit and R v4.3.0:
version information
error information
To Reproduce
I am using
precommit
in this repo: https://github.com/joelnitta/dwctaxonClone the repo, set up precommit, then try making a commit.
Expected behavior A clear and concise description of what you expected to happen.
No error
Additional context
packageVersion('renv')
: 0.17.3