Error using `install_github()`: HTTP error 401. Bad credentials. #641

Closed isaactpetersen closed 2 years ago

I'm trying to install an R package from a publicly available GitHub repo using install_github() from the {remotes} package. I'm doing this in the context of a Docker image (rocker/verse) using a GitLab runner that that will output a book to GitLab Pages using {bookdown}. However, I receive an error when trying to install an R package ({uroc}) from a publicly available GitHub repo (https://github.com/evwalz/uroc).

Here is a minimal reproducible example based on my .gitlab-ci.yml file:

variables:
  GIT_STRATEGY: clone

image: rocker/verse

.bookdown:
  stage: deploy
  script:
    - R -e "install.packages('remotes')"
    - R -e "remotes::install_github('evwalz/uroc')"

Here is the error I receive:

> remotes::install_github('evwalz/uroc')
Using bundled GitHub PAT. Please add your own PAT to the env var `GITHUB_PAT`
Error: Failed to install 'unknown package' from GitHub:
  HTTP error 401.
  Bad credentials
  Rate limit remaining: 52/60
  Rate limit reset at: 2021-08-15 17:58:54 UTC

Interestingly, I can install it fine when running the code locally, and when I run the Docker container locally, the package installs fine, as well. So it appears to be some interaction of the Docker container with the GitLab runner.

The issue appears to be similar to other issues: https://github.com/r-lib/remotes/issues/638 https://github.com/r-lib/remotes/issues/481 https://github.com/r-lib/remotes/issues/330 https://github.com/r-lib/devtools/issues/1676 https://github.com/Azure/doAzureParallel/issues/359 https://gist.github.com/Z3tt/3dab3535007acf108391649766409421

For instance, in Issue https://github.com/r-lib/remotes/issues/638, the user was able to fix the HTTP error 401 on GitHub Actions by defining GITHUB_PAT with the job's token:

    env:
      GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

The user in the other thread noted that it seems that relying on the bundled PAT is not robust. However, I'm not sure how to do that in the context of a GitLab runner (not GitHub Actions) in my .gitlab-ci.yml file. Also, please note that I'm trying to download an R package from a publicly available GitHub repo, so it shouldn't require at Personal Access Token (PAT).

Here's my sessionInfo():

> sessionInfo()
R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.2 LTS
Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so
locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=C             
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     
other attached packages:
[1] remotes_2.4.0
loaded via a namespace (and not attached):
[1] compiler_4.1.1 tools_4.1.1

Should be fixed by 77966d45c67c4212eeeb0dc1b6bbc03a12b37043

Is there an ETA of when the fix will be published in the CRAN package?

@PGijsbers you should not really be relying on the bundled PAT, either you your own PAT or use the automatically supplied GITHUB_TOKEN as a PAT, e.g.

env: 
  GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

Thanks. We're not (just) running the installation in CI so I'll need to do some additional tinkering to allow users to supply their own PAT. It still seems odd that downloading public repositories requires a PAT (this concern of the original post was not addressed).

Using the PAT is only to increase the rate limits. The default rate limit for unauthenticated requests is only 60 per hour, and remotes needs to do a API request for all packages being installed, to verify their versions and see if they need to be updated.

You can do it without the PAT but you may then find you are running into rate limits often.

Ah, that makes sense. Thanks!

@jimhester It was working for me for a while when I upgraded to {remotes} 2.4.1. However, it stopped working for me again. Using R 4.1.1 on a GitLab Runner and {remotes} 2.4.1, here's the error I receive:

> remotes::install_github('evwalz/uroc')
Using bundled GitHub PAT. Please add your own PAT to the env var `GITHUB_PAT`
Error: Failed to install 'unknown package' from GitHub:
  HTTP error 401.
  Bad credentials
  Rate limit remaining: 57/60
  Rate limit reset at: 2021-10-17 13:31:06 UTC

Note that I'm not also getting it when trying to install CRAN dependencies too:

> remotes::install_deps(dependencies = TRUE)
Error: HTTP error 401.
  Bad credentials
  Rate limit remaining: 55/60
  Rate limit reset at: 2021-10-17 13:31:06 UTC

In case it helps, here's my .gitlab-ci.yml file:

variables:
  GIT_STRATEGY: clone

image: rocker/geospatial

.bookdown:
  stage: deploy
  script:
    - mkdir public
    - R -e "install.packages('remotes')"
    - R -e "remotes::install_github('evwalz/uroc')"
    - R -e "remotes::install_deps(dependencies = TRUE)"
    - Rscript -e "bookdown::render_book(input = 'index.Rmd', output_format = 'bookdown::gitbook', output_dir = 'public')"
    - apt-get install -y rsync
    - rsync -av --delete public/ /websites/bookdown/prod/
  tags:
    - bookdown #specify name of runner
  artifacts:
    paths:
      - public

pages:
  extends: .bookdown

mr-review:
  extends: .bookdown
  after_script:
    - echo "ENVIRONMENT_URL=https://$CI_PROJECT_NAMESPACE.$CI_PAGES_DOMAIN/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/public/index.html" >> deploy.env
  artifacts:
    reports:
      dotenv: deploy.env
  environment:
    name: review/$CI_COMMIT_REF_NAME
    url: $ENVIRONMENT_URL
  only:
    - merge_requests

Here are the results of my sessionInfo():

sessionInfo()
R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.3 LTS
Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so
locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=C             
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     
other attached packages:
[1] remotes_2.4.1
loaded via a namespace (and not attached):
[1] compiler_4.1.1 tools_4.1.1

Sorry for the direct message. I wasn't sure if you'd receive it now that the issue is closed. Thanks!

Getting a similar error on a private gitlab repos:

> devtools::install_deps(quiet = TRUE)
Error: HTTP error 401.
  Bad credentials
  Rate limit remaining: 57/60
  Rate limit reset at: 2024-05-07 1[3](https://gitlab....[hiding the URL on purpose]):59:43 UTC

Execution halted
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1

Here is my .gitlab-ci.yml file:

# use the tidyverse rocker image, as it contains devtools preinstalled
image: rocker/tidyverse

# list the various jobs to be run
stages:
  - build
  - check

# define the jobs
build-job:
  stage: build
  script:
    # install
    - echo "Verifying that epirs can be built..."
    - R -e "devtools::install_deps(quiet = TRUE)"
    - R -e 'devtools::build(binary = TRUE)'
    - echo "Success!"

check-job:
  stage: check
  script:
    # run the R CMD check on the package (this path); ignore vignettes as they take time
    - echo "Running checks (no vignette) on the package..."
    - R -e 'devtools::check(vignettes = FALSE)'
    - echo "Success!"

r-lib / remotes

Error using `install_github()`: HTTP error 401. Bad credentials. #641