Closed ntrlshrp closed 6 months ago
Thanks, yes, we've previous considered this in #69, did you take a look at that? Checking again just now, I see that the developers of usethis are not interested to support gitlab at the moment: https://github.com/r-lib/usethis/issues/567
Thanks, @benmarwick, for alerting me to the very similar https://github.com/benmarwick/rrtools/issues/69. Nonetheless, that issue seemed to be resolved by a solution that does not resolve the issue I've posted here.
Solution to https://github.com/benmarwick/rrtools/issues/69 [bold added by @ntrlshrp ]:
My new issue #91:
I would like the convenience function, rrtools::use_gitlab
, to instead:
This convenience function strikes me as easy and allows maximum benefit from rrtools
usage with minimum user resources (i.e., time, decisionmaking, recoding, etc. within and across different services).
Yes, it would be useful to see this function you propose appear in your fork where we can explore and test it. Given the challenges that the usethis
developers had with the GitHub functions, I wonder it may take some time to get functions for GitLab just right.
Your proposal (a) may need further discussion. We decided with create_compendium
to capture only the first 5 steps because very few people are using the subsequent steps (docker, travis, tests), and they usually can't fit into a short workshop on rrtools.
So it might be more consistent with our current approach to have an argument in those docker and travis functions that has a default value of GitHub, which the user can change to GitLab.
Finally with some free time, I've pushed an MWE of rrtools::use_gitlab()
to my fork where others can explore and test.
Recipe Part 1
feature/gitlab
branch here: https://github.com/ntrlshrp/rrtools/tree/feature/gitlabrrtools::create_compendium("myNewPackage")
myNewPackage
project] Run `token <- readLines("path/location/of/gitlab/token")
rrtools::use_gitlab(auth_token = token)
README.Rmd
", "Overwrite CONDUCT.md
", and "Overwrite CONTRIBUTING.md
"Part 2
GitLab will run two CI/CD jobs, check-package
and render-paper
. These should both succeed. The user has now produced a reproducible paper.html
!
Part 3
R/
(e.g., hello()
) [so that coverage report works]
devtools::load_all()
and devtools::document
)myNewPackage/.gitlab-ci.yml
so that lines 8, 10, and 12 say "yes"
instead of "no"
GitLab will run five CI/CD jobs, the two jobs above plus pages
, push-image-to-registry
, and code-coverage-report
. These should all succeed and produce the following outputs, respectively.
Outputs
paper.html
is published here: https://USERname.gitlab.io/PROJECTname/myNewPackage
is published here: https://registry.gitlab.com/group/project/imageI'm curious to see if these changes work for those who primarily use github and, more generally, on other machines. There is no rigorous testing of the changes yet, simply an MWE to get things off the ground.
Details Part 1 rrtools has now:
myNewPackage
to gitlab.com/username/pkgname.git master
username
is the value from git config user.name
in Terminal and pkgname
comes from DESCRIPTION
file, line 1.gitlab-ci.yml
to myNewPackage
that, once pushed to gitlab.com, will:
devtools::check()
within Docker image rocker/{{{rocker}}}:{{{r_version}}}
paper.html
using Dockerfile
CONTRIBUTING.md
and README.Rmd/md
to point to gitlab.com instead of github.comDockerfile
to myNewPackage
that can make a Docker container using image rocker/{{{rocker}}}:{{{r_version}}}
analysis
, ^\.gitlab-ci\.yml$
, and Dockerfile
to .Rbuildignore
so the R package builds correctlyanalysis/
for template paper.Rprofile
that rrtools wroteWhen I tried to test this workflow I got stuck because my git user name does not equal my gitlab user name.
username <- system("git config user.name", intern = TRUE)
is not a sufficient solution, I guess. usethis
solves this problem for use_github()
by getting the user name from the API (git_credentials(protocol, auth_token)
). Maybe you could use something like this as well? Maybe gitlabr
could help you solve this.
Maybe you could also add some documentation for use_gitlab
mirroring the documentation of usethis::use_github
. I consider it especially important to add a sentence about the settings of the API token. I always wonder which checkboxes to click.
Thanks, @nevrome ! This is awesome. I don't have time presently, but will attempt to resolve this issue %1 where gitlab username does not equal the value returned by git config user.name
. I will also attempt documentation similar to usethis::use_github.
In response to @nevrome, who pointed out a simple, but critical bug (and the need for more documentation), I've whipped up a new branch on my fork, feature/gitlab-02
.
Recipe Part 1
remotes::install_github('ntrlshrp/rrtools', ref = 'feature/gitlab-02')
rrtools::create_compendium("myNewPackage")
rrtools::use_gitlab(auth_token = readLines("path/location/of/gitlab/personal-access-token"))
?rrtools::use_gitlab
use_gitlab()
now interactively requests your GitLab LOGIN ("GitLab username", NOT your full name). For programmatic deployment, you can set options(gitlab.username = '<GITLAB_USER_LOGIN>')
Part 2
GitLab will run two CI/CD jobs, check-package
and render-paper
. These should both succeed. The user has now produced a reproducible paper.docx!
Part 3 is as above.
NOTES:
rrtools::use_gitlab()
auth_token
?use_gitlab()
inside a git repo?origin
remote with this local repo?rrtools::use_gitlab()
alters README to be GitLab specific (this needs refinement before deployment, e.g., Binder link still says gh
instead of gl
, but shows the method I have in mind to accomplish this alteration)..gitlab-ci.yml
for the rrtools
package as a sort of external unit test for this (and other) use cases and scenarios for rrtools::use_gitlab()
.
rrtools
branch is on a GitLab project (and you've added your personal access token as a variable, RRTOOLS_TOKEN
, to this project) and you trigger a CI/CD pipeline (e.g., Click "Edit" on .gitlab-ci.yml
and then click "Commit"), the pipeline runs a rocker/verse
container, installs an rrtools
branch that includes use_gitlab()
, creates a new compendium as if it were a user, and runs rrtools::use_gitlab()
within that compendium. This associates the local repo (inside rocker/verse
) with a new remote repo on GitLab.com, https://gitlab.com/$GITLAB_USER_LOGIN/newRRtoolsCompendium..gitlab-ci.yml
derived from the rrtools/inst/gitlab-ci.yml
template. This runs two CI/CD jobs (see Part 2 above), check-package
and render-paper
. These should both succeed. The user has now produced a reproducible paper.docx
!rocker/verse
(which probably makes more sense in the .travis.yml
).pages
, push-image-to-registry
, and code-coverage-report
.Feedback:
I hope this is helpful. Thanks.
PS: Running Parts 1 and 2 inside a Docker container (replace <GITLAB_USER_LOGIN>
and <your-personal-access-token>
appropriately).
Dockerfile
FROM rocker/verse:3.5.3
RUN . /etc/environment \
&& R -e "remotes::install_github('ntrlshrp/rrtools', ref = 'feature/gitlab-02')" \
&& cd / \
&& git config --global user.name "MyGitUSERNAME" \
&& git config --global user.email "MyGitUSEREMAIL" \
&& R -e "rrtools::create_compendium('newRRtoolsCompendiumDocker')" \
&& cd newRRtoolsCompendiumDocker \
&& R -e "options(gitlab.username = '<GITLAB_USER_LOGIN>'); rrtools::use_gitlab(auth_token = '<your-personal-access-token>')" \
&& git add . \
&& git commit -m "Added all new files since initial commit" \
&& git push https://oauth2:<your-personal-access-token>@gitlab.com/<GITLAB_USER_LOGIN>/newRRtoolsCompendiumDocker.git
Thank you for your patience, @ntrlshrp. I finally came around to test this fixed implementation. Everything seems to work and the documentation is fine. You did an excellent job here and I'm sorry that it took my so long to come back to this.
What do you think @benmarwick?
I suggest you, @ntrlshrp, create PR to the current rrtools master and we review it formally. You don't have to add any message in the PR, just a link to this issue. If more tests and features are required it will become clear there.
I'm for example not sure if these backup .bak-files (?) are necessary. I also get the warning
Warning messages:
1: In file.rename("CONDUCT.md", "CONDUCT.md.bak") :
cannot rename file 'CONDUCT.md' to 'CONDUCT.md.bak', reason 'No such file or directory'
2: In file.rename("CONTRIBUTING.md", "CONTRIBUTING.md.bak") :
cannot rename file 'CONTRIBUTING.md' to 'CONTRIBUTING.md.bak', reason 'No such file or directory'
when I apply use_gitlab()
in a new repo. Probably this backup creation should be wrapped in if (file.exists(...)) {}
. But these are minor details we can easily discuss/add later.
No need for apologies, @nevrome , glad to hear that it looks ready for a more formal review. Please see the suggested PR. Thanks again!
Also, your solution to the backup .bak
files sounds ideal. My use case started with rrtools::create_compendium()
but if the user has not directly or indirectly run the rrtools::use_readme_rmd()
function before rrtools::use_gitlab()
, the CONDUCT
and CONTRIBUTING
files likely aren't there and thus don't need backing up. In any event, I'll wait for more issues to pop out of the formal review before making new commits. Thanks again!
Closing due to lack of activity.
I'm curious to see if others think this would be valuable to this package. Thanks!
Is your feature request related to a problem? Please describe. I want to use
rrtools
and host my remote repository on GitLab, not GitHub. Currently, there are no convenience functions to make this happen (nor to change all instances ofgithub.com
togitlab.com
(e.g., in theREADME.Rmd
)).Describe the solution you'd like Offer users a convenience function,
rrtools::use_gitlab
that is analogous tousethis::use_github(credentials = git2r::cred_ssh_key(), auth_token = "xxxx", protocol = "https", private = FALSE)
doing the following:git push --set-upstream
git remote add origin
.gitlab-ci.yml
(GitLab does their own Continuous Integration and Continuous Development (CI/CD))Dockerfile
in that.gitlab-ci.yml
(so you can reproduce the outputs usingdocker
rather thangitlab-runner
)Outside this convenience function, other changes may be necessary:
rrtools::use_travis()
function to alert the user to possible duplication and allow an off-ramp if GitLab CI/CD already covers the user's needsrrtools::create_compendium()
function to ask if GitHub or GitLabREADME.Rmd
onrrtools
to alert people that you can dorrtools::use_gitlab()
to do in a single step what is currently represented as steps (3b)usethis::use_github
, (6)rrtools::use_dockerfile()
, and (7)rrtools::use_travis()
Describe alternatives you've considered I could attempt to request this feature in the
usethis
package rather than here, i.e.,usethis::use_gitlab
. My sense, however, is that this is fairly simple to start a new repository by simply calling somesystem()
functions although, at the same time, I see thatusethis::use_github
is ~70 lines with at least 6 checks that the function can appropriately succeed.Also, if
rrtools
uses the hypotheticalusethis::use_gitlab()
, it may be difficult to wrap into a single step what is currently represented as steps (3b)usethis::use_github
, (6)rrtools::use_dockerfile()
, and (7)rrtools::use_travis()
.Additional context NA