Closed vidavidorra closed 4 years ago
rarkins
commented
4 years ago Sorry, it’s not possibly for the app to run with a PAT. You would need to run your own copy of Renovate with a PAT (possibly just for actions only if you want to keep the app for everything else).
vidavidorra
commented
4 years ago So you mean a self-hosted Renovate is the only option to have renovate updating GitHub actions?
rarkins
commented
4 years ago So you mean a self-hosted Renovate is the only option to have renovate updating GitHub actions?
Yes, because the hosted app uses app tokens and GitHub forbids app tokens from changing github workflows
vidavidorra
commented
4 years ago Too bad it is not supported, but guess I'll give self-hosting a go. I've looked though the docs I could find on it and think the following would work, but would appreciate confirmation/feedback if you please.
Some follow up questions that are not quite clear from those docs.
config.js configuration supplied to the container?renovate.json like this. Does the self-hosted instance use the same configuration and how would it know what repositories to run on?rarkins
commented
4 years ago
- Is the documentation described here for the
config.jsconfiguration supplied to the container?
You'd need to make sure the container you run in actions has access to it, if that's how you plan to configure it.
- How would I configure to only use this for updating GitHub actions?
See the enabledManagers setting
3. How is this configured for repositories to run on? Normally Renovate runs on a repository and all config is using the
renovate.jsonlike this . Does the self-hosted instance use the same configuration and how would it know what repositories to run on?
You would need to use a PAT that has access to all the repositories it needs to run on. Then either (a) use the autodiscover field if you want it to run on all of them, or (b) configure the list of repositories in the command you use to run the renovate action
vidavidorra
commented
4 years ago You'd need to make sure the container you run in actions has access to it, if that's how you plan to configure it.
Yeah, I plan to have a separate repository for this self-hosted renovate, lets say a repo named .renovate. This repository can than have a file config.js, which can be mounted to the container.
Was the documentation I linked there what can be configured in the config.js file though?
Is that how you'd recommend to run this or do you think running the npm/docker in the specific repo I want to update the actions of as a separate workflow would be better?
See the enabledManagers setting
That is from the general configuration options (here) so that would be the config of the repo I want to update the actions of right? Let's say I've got the next-standard-version repository of which I want to update the actions and a .renovate which is running the renovate/renovate Docker. This configuration for enabledManagers would be in the next-standard-version config right?
rarkins
commented
4 years ago Was the documentation I linked there what can be configured in the
config.jsfile though?
yes
Is that how you'd recommend to run this or do you think running the npm/docker in the specific repo I want to update the actions of as a separate workflow would be better?
I think you're better to have a single repo updating all the others instead of you needing to add actions to each of them separately.
That is from the general configuration options (here) so that would be the config of the repo I want to update the actions of right?
Assuming that you want to keep the app for all other managers and use a cron'd action just for github actions, you'd want enabledManagers to be on the bot config, not repo config - otherwise it would take effect with the app too.
Also you need to make sure you set a different branchPrefix for the actions bot, so that it doesn't conflict with the hosted app.
Finally, strongly recommend you test this out on a dummy repository first. e.g. have npm and actions, and install the app to confirm it's making PRs for npm. Then set up your actions bot and confirm it makes a PR for the action without deleting the app's PR for npm.
vidavidorra
commented
4 years ago Thanks for the help @rarkins! I'll give this a shot and report back results here. Will probably look into it this weekend and try to set it up.
rarkins
commented
4 years ago A blog post or case study would be great if you can take notes. we can add it to docs ot publish it on our blog if you don't have your own
vidavidorra
commented
4 years ago Will do my best to take notes ;)
Happy to have it added to the docs and/or blog. I don't have a blog myself but would be happy writing something up for your guys' blog once I've figured everything out.
stale[bot]
commented
4 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed soon if no further activity occurs.
vidavidorra
commented
4 years ago I've played around with this today, but can't quite get it to work. So far this is local using the docker container and the config.js that I mount to the container. The container finds the config and accepts it (I previously had issues with it but the error message of the validation helped me figure it out).
This is my repo that I use for testing: https://github.com/vidavidorra/.github-renovate-test
I see the logs that it detects my github-actions workflows, but it doesn't seem to do anything with it.
Here's the log where I added "enabledManagers": ["github-actions"], but without that in the repo's log the output is the same.
Why aren't the workflows analysed (i.e. I don't see it detecting this action that is deliberately an old version)?
DEBUG: git-submodules is not in enabledManagers list - skipping (repository=vidavidorra/.github-renovate-test)
DEBUG: Using file match: ^\.github/main.workflow$ for manager github-actions (repository=vidavidorra/.github-renovate-test)
DEBUG: Using file match: ^\.github/workflows/[^/]+\.ya?ml$ for manager github-actions (repository=vidavidorra/.github-renovate-test)
DEBUG: Matched 3 file(s) for manager github-actions: .github/workflows/lint-commit-messages.yml, .github/workflows/lint.yml, .github/workflows/release.yml (repository=vidavidorra/.github-renovate-test)
DEBUG: github-actions.extractPackageFile() (repository=vidavidorra/.github-renovate-test)
DEBUG: github-actions.extractPackageFile() (repository=vidavidorra/.github-renovate-test)
DEBUG: github-actions.extractPackageFile() (repository=vidavidorra/.github-renovate-test)
DEBUG: gitlabci is not in enabledManagers list - skipping (repository=vidavidorra/.github-renovate-test)With the full log here:
And here my config.js
module.exports = {
platform: 'github',
token: 'xxx',
logFileLevel: 'warn',
logLevel: 'debug',
onboarding: false,
enabledManagers: ['github-actions'],
trustLevel: 'high',
dryRun: true,
gitAuthor: 'Renovate Bot <bot@renovateapp.com>',
repositories: ['vidavidorra/.github-renovate-test'],
};
viceice
commented
4 years ago Found it:
We currently only support renovating docker images
viceice
commented
4 years ago So feel free to open a new issue feature request in our main repo for supporting other actions.
Should be easy to implement, cause we can reuse github-tags datasource.
As a workaround you can use the regex manager to update them:
{
...
regexManagers: [
{
fileMatch: ['^\\.github/workflows/[^/]+\\.ya?ml$'],
matchStrings: ['uses: (?<depName>.*?)@(?<currentValue>.*?)\\s'],
datasourceTemplate: 'github-tags',
},
],
...
}So updating the workflows is supported, but within the workflow only Docker images are updated. Did I understood that correctly?
So in the example below, the container would be updated if there was a newer version (with the regular Docker update rules of course).
jobs:
stuff:
runs-on: ubuntu-latest
container: vidavidorra/docker-linux:ubuntu1804-masterSo feel free to open a new issue feature request in our main repo for supporting other actions.
Will do, is that the repo: https://github.com/renovatebot/renovate ?
As a workaround you can use the regex manager to update them:
Where would that need to be specified? I just tried adding it to my config.js, the config that I use for running the Renovate Docker image but that didn't change anything. Should that be in the other repo (of which I want to update the actions) then?
Another question: How can I configure the Renovate Docker that I'm running to only update GitHub Actions? I've currently got enabledManagers: ['github-actions'], set in the config.js, but that is not an actual supported self-hosted config according to the docs. I think I'd need to have a renovate.json like config that I can pass to the Docker container and that would be merged with the config from the target repository. Is that correct and how can that be achieved?
viceice
commented
4 years ago So feel free to open a new issue feature request in our main repo for supporting other actions.
Will do, is that the repo: https://github.com/renovatebot/renovate ? yes
As a workaround you can use the regex manager to update them:
Where would that need to be specified? I just tried adding it to my
config.js, the config that I use for running the Renovate Docker image but that didn't change anything. Should that be in the other repo (of which I want to update the actions) then?Another question: How can I configure the Renovate Docker that I'm running to only update GitHub Actions? I've currently got
enabledManagers: ['github-actions'],set in theconfig.js, but that is not an actual supported self-hosted config according to the docs. I think I'd need to have arenovate.jsonlike config that I can pass to the Docker container and that would be merged with the config from the target repository. Is that correct and how can that be achieved?
you need to enable regex manager too.
You can add any normal config options to the config.js can you show the new config and the log?
vidavidorra
commented
4 years ago This is my new log and config. I've now removed the enabledManagers and regexManagers from the repo and added that to my config.js.
In the logs it looks like this will indeed update the action that I've deliberately put on an old version (wagoid/commitlint-github-action).
Thanks for your help so far. Tonight I'll try and set this up to run from my .github-renovate repo on a scheduler etc. (I'm currently running the Docker container locally). And also run this with dryRun: false to see whether it indeed creates the PR to update this.
config.js
module.exports = {
platform: 'github',
token: 'xxx',
logFileLevel: 'warn',
logLevel: 'debug',
onboarding: false,
enabledManagers: ['github-actions', 'regex'],
trustLevel: 'high',
dryRun: true,
gitAuthor: 'Renovate Bot <bot@renovateapp.com>',
repositories: ['vidavidorra/.github-renovate-test'],
regexManagers: [
{
fileMatch: ['^\\.github/workflows/[^/]+\\.ya?ml$'],
matchStrings: ['uses: (?<depName>.*?)@(?<currentValue>.*?)\\s'],
datasourceTemplate: 'github-tags',
},
],
};And the log:
vidavidorra
commented
4 years ago Also, this is the config of the repo: https://github.com/vidavidorra/.github-renovate-test/blob/master/.github/renovate.json
{
"extends": [
"config:base",
":pinDependencies",
":timezone(Europe/Amsterdam)",
":assignee(vidavidorra)"
],
"prHourlyLimit": 0
}looks good to me now, found one update.
vidavidorra
commented
4 years ago I've ran the (local) renovate Docker, without dryRun this time, and it generated a PR for me. Awesome!
However, shortly after the PR was generated, the Renovate App seems to have closed this PR (see vidavidorra/.github-renovate-test#13.
Could that be due to not having included a different branchPrefix in my config.js?
viceice
commented
4 years ago Yes, you need to use a different branch prefix
vidavidorra
commented
4 years ago I've got it working without interference with the Renovate App. I'm currently working on the setup with the container in my separate repo for this.
One thing I noted is that these updates don't have a depType (I'd like to schedule these updates from the target repo). Is that a bug or do some updates don't have an explicit type? Before this I've only seen these logs for npm packages, which includes the depType as far as I've seen, so I'm not familiar with other updaters.
DEBUG: packageFiles with updates (repository=vidavidorra/.github-renovate-test)
"config": {
"regex": [
{
"packageFile": ".github/workflows/lint-commit-messages.yml",
"manager": "regex",
"deps": [
{
"depName": "actions/checkout",
"currentValue": "v1",
"datasource": "github-tags",
"autoReplaceData": {
"depIndex": 0,
"replaceString": "uses: actions/checkout@v1\n"
},
"updates": [],
"warnings": [],
"skipReason": "unsupported-value"
},
{
"depName": "wagoid/commitlint-github-action",
"currentValue": "v1.4.0",
"datasource": "github-tags",
"autoReplaceData": {
"depIndex": 1,
"replaceString": "uses: wagoid/commitlint-github-action@v1.4.0\n"
},
"updates": [
{
"fromVersion": "v1.4.0",
"toVersion": "v1.6.0",
"newValue": "v1.6.0",
"newMajor": 1,
"newMinor": 6,
"updateType": "minor",
"isSingleVersion": true
}
],
"warnings": [],
"sourceUrl": "https://github.com/wagoid/commitlint-github-action"
}
],
"matchStrings": ["uses: (?<depName>.*?)@(?<currentValue>.*?)\\s"],
"autoReplace": true
},
...nvm.
After some more looking around I've found that adding rules for a specific manager will work neatly.
"packageRules": [
{
"managers": ["github-actions", "regex"],
"schedule": ["before 6am"]
}
]Apologies for the open and close. I'm still working on this, but am getting nearer to a working thing!
stale[bot]
commented
4 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed soon if no further activity occurs.
vidavidorra
commented
4 years ago Just an update on this: I've mostly got it working with a very easy setup. I'm in multiple organisations and a PAT will give acces to all those repo's as well, so I'm changing some stuff in my repo/org setup before I can definitively start running this with a PAT with write access. So far I'm only doing tests with read-only access and dry-run enabled, and all looks to be working perfectly so far!
When I've got my PAT setup (somewhere this week) and Renovate running I'll report back here with setup and results and then we can also discuss inclusion in docs and/or blog post.
jdbruijn
commented
4 years ago I've got this fully up and running now!
The repo where I run this on: https://github.com/vidavidorra/github-renovate And the repo where I've tested a few things on: https://github.com/vidavidorra/.github-renovate-test
So far I've verified that it creates PRs and rebases them if instructed to do so. As this is (I believe) more or less the same program you guys use for the app I'll leave it at that for the tests.
How would you like the write-up on this? I'm thinking a section in the docs (perhaps in the Getting Started) will be the easiest to find for users. And maybe a blog post with this case too?
Some parts I want to include in it are:
rarkins
commented
4 years ago Great! Can you write it up here and then we work out the best place to host the content?
jdbruijn
commented
4 years ago @rarkins Appologies this all took so long, but here it is. I hope everything is clear from this, but if I missed something please let me know. I've taken parts of this setup from my two repositories where I created the action and run the self-hosted Renovate (github-action-renovate and github-renovate). I recommend referencing the last as example of how this is used/set up.
Renovate needs a personal access token to access the repositories to run on. This token can have just the repo:public_repo scope for public repositories. For private repositories, the repo scope is required.
The configuration is done using a config.js file that contains the self-hosted configuration options, but may also contain the regular configuration options. An example config is shown below, which is configured to only update GitHub workflow actions. In this configuration it is important to include the branchPrefix option to prevent interference with e.g. the GitHub Renovate App. The regexManagers is used to update the actions and I've created an issue (#5733) to also support actions with just the github-actions manager.
const branchName = 'github-renovate';
module.exports = {
branchPrefix: `${branchName}/`,
enabledManagers: ['github-actions', 'regex'],
gitAuthor: 'Renovate Bot <bot@renovateapp.com>',
logLevel: 'debug',
onboarding: true,
onboardingBranch: `${branchName}/configure`,
platform: 'github',
regexManagers: [
{
datasourceTemplate: 'github-tags',
fileMatch: ['^\\.github/workflows/[^/]+\\.ya?ml$'],
matchStrings: ['uses: (?<depName>.*?)@(?<currentValue>.*?)\\s'],
},
],
repositories: [
'vidavidorra/github-action-renovate',
'vidavidorra/next-standard-version',
],
};Lastly, the workflow to run all this is triggered on a schedule to frequently check if anything needs to be updated. This is a little bit less responsive, especially when rebasing a PR, then e.g. using the GitHub Renovate App since this is bound to a schedule. Therefore, I highly recommend to only use this where the GitHub Renovate App can not be used, like for updating workflows. It could be ran on a schedule as frequently as five minutes, but a schedule of fifteen minutes has been working fine for me so far.
name: Renovate
on:
schedule:
# The "*" (#42, asterisk) character has special semantics in YAML, so this
# string has to be quoted.
- cron: '0/15 * * * *'
jobs:
renovate:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2.0.0
- name: Renovate
uses: vidavidorra/github-action-renovate@v1.0.2
with:
configurationFile: src/config.js
token: ${{ secrets.RENOVATE_TOKEN }}Just to let you know I had my first "production" updates ran by this setup: https://github.com/vidavidorra/github-action-renovate/pull/17 :smile:
viceice
commented
4 years ago @rarkins Maybe we should add such a cron for our own repos, so we can pin our actions too?
rarkins
commented
4 years ago Yes, I think we should. Perhaps we can host it in the github.com/renovatebot/* domain too if @jdbruijn is ok with that? We would list you as original author of course. This way we can include it in our "official" way to use Renovate and it should be more automatically trusted by people.
jdbruijn
commented
4 years ago Perhaps we can host it in the github.com/renovatebot/* domain too if @jdbruijn is ok with that?
You mean that https://github.com/vidavidorra/github-action-renovate would become an "official" action in the github.com/renovatebot/* domain? (That is moving that action to Renovate and listing that as method to update GA workflows.)
rarkins
commented
4 years ago Yes I was thinking maybe github.com/renovatebot/github-action and we support + document + recommend it officially as a way to run Renovate in actions to update actions. What do you think?
jdbruijn
commented
4 years ago I think that is awesome! Would love to have this (little project it has become) supported officially and have more people use it! So I think it is great.
jdbruijn
commented
4 years ago I just want to note that it is even more powerful than that. It can also be used to run certain tasks that require thrustLevel=high like postUpgradeTasks!
rarkins
commented
4 years ago Great point. The GitHub Action may grow in usefulness over time including as GitHub make Actions themselves more capable. Can you add me as an admin to your repo so that I have permissions to transfer it into the renovatebot org?
jdbruijn
commented
4 years ago Can you add me as an admin to your repo so that I have permissions to transfer it into the renovatebot org?
Yes, but I need a little bit of time before doing so. I need to disable my repository using this action so that doesn't fail. I can add you later today.
I've also got a couple of questions.
jdbruijn
commented
4 years ago @rarkins I'm ready for the transfer, could you please respond to my previous comment and then I'll add you to the repo!
rarkins
commented
4 years ago
- I still want to be able to contribute to this project and help keep improving it. Would that be possible?
Absolutely, it will be open source and available for PRs like normal. I'd also like to acknowledge you as the primary author in the package.json as well as in the README
- When do you think the action will be available from Renovate (i.e. this project moved to Renovate and available as action in the GitHub Marketplace)? I need to know this so I know how long my updates are down on my side.
I'm not actually familiar with how long this takes. I thought that being in the marketplace was not a pre-requisite for using it, but I'm not sure.
It's possible that it won't be necessary though. I'm testing out if simply importing your repository works just as well, so you can keep both in parallel for a while and later Archive yours if you wish. I also then don't need any admin rights to yours. You can see the result here: https://github.com/renovatebot/github-action @viceice any concerns with this approach instead?
All your commits are preserved, etc.
- I also want to keep this open source, but think it will be as the Renovate products are open source as well. Can you confirm that this is kept open source please?
Confirmed
- Also, I want to let you know that you may convert the license to another OSS license (e.g. the same license you use for other Renovate projects).
Thanks, that would be ideal as it uses Renovate as a dependency so should use the same license.
viceice
commented
4 years ago I'm fine with the copy approach.
Next Steps:
renovate ?viceice
commented
4 years ago so when we finished, @jdbruijn can deprecate his action
jdbruijn
commented
4 years ago Absolutely, it will be open source and available for PRs like normal. I'd also like to acknowledge you as the primary author in the package.json as well as in the README
Sounds good to me!
I'm not actually familiar with how long this takes. I thought that being in the marketplace was not a pre-requisite for using it, but I'm not sure.
According to the docs I think you might be right. https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsuses It doesn't explicitly state anything about the marketplace. To be honest, publishing to the marketplace is a bit of a pain in the ass as you have to do that step manually.
It's possible that it won't be necessary though. I'm testing out if simply importing your repository works just as well, so you can keep both in parallel for a while and later Archive yours if you wish. I also then don't need any admin rights to yours. You can see the result here: https://github.com/renovatebot/github-action
Even better, looks good! Once that is available to use I'll add a note to my repository refer possible users to the Renovate repository/docs and archive my repository as that would no longer be needed!
jdbruijn
commented
4 years ago The example needs a GitHub secret RENOVATE_TOKEN. I was just using a PAT with no specific rights configured so it has read-only rights ;)
jdbruijn
commented
4 years ago And the config probably needs to be updated to point to renovatebot/github-action. https://github.com/renovatebot/github-action/blob/master/example/config.js#L8
rarkins
commented
4 years ago Thanks @jdbruijn! I have made some quick updates to align the license and update the repositories. You have write access, so feel free to clone that repo and raise PRs for any other changes you can immediately see. Let's work in that repo from now on, so I'll close this issue.
jdbruijn
commented
4 years ago Sounds good!
Which Renovate are you using?
WhiteSource Renovate App
Which platform are you using?
GitHub.com
Have you checked the logs? Don't forget to include them if relevant
Not relevant
What would you like to do?
I just saw the support for GitHub actions updating in the docs, which is awesome! I want to set that up, but saw that it needs a Personal Access Token instead of using the app (which I currently am using).
I've searched the docs but couldn't find how to setup renovate with a personal acces token. I've only found something for GitLab (here) but am not sure if the same applies to GitHub. Could you please point me in the right direction to setup renovate with a personal access token?
Another question is what the downsides are of using a personal access token rather than the app? One thing I've found so far is that the personal access token is for all repo's and the app can be configured for specific repo's. What are other downsides of using the personal access token over the app, any functionalities only available with the app for example?