search

jycouet / VSTSExtensions

MIT License
30 stars 11 forks source link

Renovate not actually running after successful config validation #75

Closed Polymeta closed 3 months ago

Polymeta commented 3 months ago

Describe the bug

Renovate does not run anymore once the config has been (successfully) validated. See logs below

To Reproduce Steps to reproduce the behavior:

  1. Run Renovate via the RenovateMe extension V1.0.4

Expected behavior

I expect Renovate to run as it did before the config validation was introduced if the configuration got successfully validated.

Screenshots

Below some logs, I cropped out unnescary information:

2024-04-08T13:55:32.2555308Z ==============================================================================
2024-04-08T13:55:32.2555444Z Task         : Renovate Me
2024-04-08T13:55:32.2555513Z Description  : Renovate your libs on your selected Azure Repo
2024-04-08T13:55:32.2555644Z Version      : 1.0.4
2024-04-08T13:55:32.2555708Z Author       : Jean-Yves COUET
2024-04-08T13:55:32.2555789Z Help         : [Renovate Me](https://marketplace.visualstudio.com/items?itemName=jyc.vsts-extensions-renovate-me) based on [renovate](https://github.com/singapore/renovate)
2024-04-08T13:55:32.2555992Z ==============================================================================
2024-04-08T13:55:33.0698504Z ##[debug]Agent environment resources - Disk: / Available 20540.00 MB out of 74244.00 MB, Memory: Used 592.00 MB out of 6921.00 MB, CPU: Usage 16.91%
2024-04-08T13:55:33.9669258Z ##[debug]agent.TempDirectory=/home/vsts/work/_temp
2024-04-08T13:55:33.9680500Z ##[debug]loading inputs and endpoints
2024-04-08T13:55:33.9686469Z ##[debug]loading INPUT_RENOVATEOPTIONSVERSION
2024-04-08T13:55:33.9700330Z ##[debug]loading INPUT_RENOVATEOPTIONSARGS
2024-04-08T13:55:33.9703222Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2024-04-08T13:55:33.9706459Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2024-04-08T13:55:33.9711800Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2024-04-08T13:55:33.9714141Z ##[debug]loading SECRET_AZURE_PAT
2024-04-08T13:55:33.9716005Z ##[debug]loading SECRET_GITHUB_COM_TOKEN
2024-04-08T13:55:33.9720246Z ##[debug]loaded 7
2024-04-08T13:55:33.9725568Z ##[debug]Agent.ProxyUrl=undefined
2024-04-08T13:55:33.9729136Z ##[debug]Agent.CAInfo=undefined
2024-04-08T13:55:33.9729523Z ##[debug]Agent.ClientCert=undefined
2024-04-08T13:55:33.9729910Z ##[debug]Agent.SkipCertValidation=undefined
2024-04-08T13:55:33.9763455Z ##[debug]SYSTEMVSSCONNECTION auth param ACCESSTOKEN = ***
2024-04-08T13:55:33.9764281Z ##[debug]renovateOptionsVersion=latest
2024-04-08T13:55:33.9774800Z ##[debug]renovateOptionsArgs=--host-rules="omitted" --npmrc="omitted"
2024-04-08T13:55:33.9776079Z ##[debug]Validating renovate config
2024-04-08T13:55:33.9776431Z ##[debug]Renovate run n° 1
2024-04-08T13:55:33.9776776Z ##[debug]which 'npx'
2024-04-08T13:55:33.9777205Z ##[debug]found: '/opt/hostedtoolcache/node/18.20.0/x64/bin/npx'
2024-04-08T13:55:33.9777685Z ##[debug]/opt/hostedtoolcache/node/18.20.0/x64/bin/npx arg: --yes --package renovate@latest -- renovate-config-validator
2024-04-08T13:55:33.9783363Z ##[debug]/opt/hostedtoolcache/node/18.20.0/x64/bin/npx arg: --yes --package renovate@latest -- renovate-config-validator
2024-04-08T13:55:33.9785474Z ##[debug]exec tool: /opt/hostedtoolcache/node/18.20.0/x64/bin/npx
2024-04-08T13:55:33.9785950Z ##[debug]exec tool: /opt/hostedtoolcache/node/18.20.0/x64/bin/npx
2024-04-08T13:55:33.9787305Z ##[debug]arguments:
2024-04-08T13:55:33.9787664Z ##[debug]arguments:
2024-04-08T13:55:33.9788002Z ##[debug]   --yes
2024-04-08T13:55:33.9798419Z ##[debug]   --yes
2024-04-08T13:55:33.9799776Z ##[debug]   --package
2024-04-08T13:55:33.9801713Z ##[debug]   --package
2024-04-08T13:55:33.9802376Z ##[debug]   renovate@latest
2024-04-08T13:55:33.9803064Z ##[debug]   renovate@latest
2024-04-08T13:55:33.9803409Z ##[debug]   --
2024-04-08T13:55:33.9805205Z ##[debug]   --
2024-04-08T13:55:33.9805550Z ##[debug]   renovate-config-validator
2024-04-08T13:55:33.9805926Z ##[debug]   renovate-config-validator
2024-04-08T13:55:33.9806692Z [command]/opt/hostedtoolcache/node/18.20.0/x64/bin/npx --yes --package renovate@latest -- renovate-config-validator
2024-04-08T13:55:33.9806870Z 
2024-04-08T13:57:46.1033237Z  INFO: Validating renovate.json
2024-04-08T13:57:46.1033462Z 
2024-04-08T13:57:46.1175357Z DEBUG: Checking for config file in config.js
2024-04-08T13:57:46.1175588Z 
2024-04-08T13:57:46.1182121Z DEBUG: No config file found on disk - skipping
2024-04-08T13:57:46.1182320Z 
2024-04-08T13:57:46.1187182Z  INFO: Config validated successfully
2024-04-08T13:57:46.1187318Z 
2024-04-08T13:57:46.5591256Z ##[debug]Exit code 0 received from tool '/opt/hostedtoolcache/node/18.20.0/x64/bin/npx'
2024-04-08T13:57:46.5593119Z ##[debug]Exit code 0 received from tool '/opt/hostedtoolcache/node/18.20.0/x64/bin/npx'
2024-04-08T13:57:46.5606568Z ##[debug]STDIO streams have closed for tool '/opt/hostedtoolcache/node/18.20.0/x64/bin/npx'
2024-04-08T13:57:46.5607403Z ##[debug]STDIO streams have closed for tool '/opt/hostedtoolcache/node/18.20.0/x64/bin/npx'
2024-04-08T13:57:46.5633432Z ##[debug]renovateArgs to run: "My Cool app" --platform azure --endpoint omitted --token *** --host-rules="omitted" --npmrc="omitted"
2024-04-08T13:57:46.5635612Z ##[debug]Run renovate
2024-04-08T13:57:46.5638731Z ##[debug]Renovate done
2024-04-08T13:57:46.5641575Z ##[debug]task result: Succeeded
2024-04-08T13:57:46.5653708Z ##[debug]Processed: ##vso[task.complete result=Succeeded;]
2024-04-08T13:57:46.5702701Z ##[section]Finishing: Renovate

Desktop (please complete the following information):

Additional context

As you can see in the logs, the config is valid and you can even see in the debug statements from this extension that renovate is supposed to run, but never actually does, otherwise we should see the typical renovate logs and the npx command being printed out in the logs, which is not the case.

My guess that this is a bug that got introduced with the config validation. The exec function uses the needToRun variable https://github.com/jycouet/VSTSExtensions/blob/04c9372d8ac1e481df80f7784989b9d14af38538/renovateMe/buildtask/src/index.ts#L80C1-L80C22 which is true to default, however during the first task (now config validation) it gets set to false and never back to true (unless automerging PRs is happening to trigger a re-run that is). Before that was never an issue because there was only one task (actually running renovate), but now with two, there needs to be some extra logic to set the needToRun variable back to true allowing renovate to run again,

Even if I'm wrong, I hope the provided log provides enough pointers to address this issue, thanks in advance!

jycouet commented 3 months ago

Thx a lot for these details, maybe @KaVeKa can also comment?

KaVeKa commented 3 months ago

@Polymeta That is indeed also what I found out. I did not know of this needToRun variable however. So thank you for describing this in detail.

I'm sorry for introducing this bug and impacting other users of this extension.

However at the moment I do not have much spare time to work on this myself, so maybe reverting the change is the fastest and safest for now to minimize impact? Fyi: I did test today that the azure step is correctly failing when the renovate.json config is invalid.

jycouet commented 3 months ago

I just published 1.0.5 right now that should fix the issue. Let me know if it's working.

(and please close the issue if it's working well)

KaVeKa commented 3 months ago

I can confirm that renovate is running again with the new version. However, even with the validation in place. It seems that my initial error case is not yet solved by using renovate's validate command.

##[debug]Evaluating condition for step: 'Run Renovate'
##[debug]Evaluating: SucceededNode()
##[debug]Evaluating SucceededNode:
##[debug]=> True
##[debug]Result: True
Starting: Run Renovate
==============================================================================
Task         : Renovate Me
Description  : Renovate your libs on your selected Azure Repo
Version      : 1.0.5
Author       : Jean-Yves COUET
Help         : [Renovate Me](https://marketplace.visualstudio.com/items?itemName=jyc.vsts-extensions-renovate-me) based on [renovate](https://github.com/singapore/renovate)
==============================================================================
##[debug]Using node path: /home/vsts/agents/3.236.1/externals/node16/bin/node
##[debug]agent.TempDirectory=/home/vsts/work/_temp
##[debug]loading inputs and endpoints
##[debug]loading INPUT_RENOVATEOPTIONSVERSION
##[debug]loading INPUT_RENOVATEOPTIONSARGS
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
##[debug]loading SECRET_HIP_NEXUS_PASSWORD
##[debug]loaded 7
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]SYSTEMVSSCONNECTION auth param ACCESSTOKEN = ***
##[debug]renovateOptionsVersion=latest
##[debug]renovateOptionsArgs=...
##[debug]Validating renovate config
##[debug]Renovate run n° 1
##[debug]which 'npx'
##[debug]found: '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]/opt/hostedtoolcache/node/20.12.0/x64/bin/npx arg: --yes --package renovate@latest -- renovate-config-validator
##[debug]/opt/hostedtoolcache/node/20.12.0/x64/bin/npx arg: --yes --package renovate@latest -- renovate-config-validator
##[debug]exec tool: /opt/hostedtoolcache/node/20.12.0/x64/bin/npx
##[debug]exec tool: /opt/hostedtoolcache/node/20.12.0/x64/bin/npx
##[debug]arguments:
##[debug]arguments:
##[debug]   --yes
##[debug]   --yes
##[debug]   --package
##[debug]   --package
##[debug]   renovate@latest
##[debug]   renovate@latest
##[debug]   --
##[debug]   --
##[debug]   renovate-config-validator
##[debug]   renovate-config-validator
/opt/hostedtoolcache/node/20.12.0/x64/bin/npx --yes --package renovate@latest -- renovate-config-validator
INFO: Validating renovate.json

 INFO: Config validated successfully

##[debug]Exit code 0 received from tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]Exit code 0 received from tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]STDIO streams have closed for tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]STDIO streams have closed for tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]renovateArgs to run: ...
##[debug]Run renovate
##[debug]Renovate run n° 1
##[debug]which 'npx'
##[debug]found: '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]/opt/hostedtoolcache/node/20.12.0/x64/bin/npx arg: renovate@latest ...
##[debug]/opt/hostedtoolcache/node/20.12.0/x64/bin/npx arg: renovate@latest ...
##[debug]exec tool: /opt/hostedtoolcache/node/20.12.0/x64/bin/npx
##[debug]exec tool: /opt/hostedtoolcache/node/20.12.0/x64/bin/npx
##[debug]arguments:
##[debug]arguments:
##[debug]   renovate@latest
##[debug]   renovate@latest
##[debug]   ...
##[debug]   ...
##[debug]   --platform
##[debug]   --platform
##[debug]   azure
##[debug]   azure
##[debug]   --endpoint
##[debug]   --endpoint
##[debug]   ...
##[debug]   ...
##[debug]   --token
##[debug]   --token
##[debug]   ***
##[debug]   ***
##[debug]   --host-rules=...
##[debug]   --host-rules=...
/opt/hostedtoolcache/node/20.12.0/x64/bin/npx renovate@latest ...

##[debug]Agent environment resources - Disk: / Available 20390.00 MB out of 74244.00 MB, Memory: Used 864.00 MB out of 6921.00 MB, CPU: Usage 3.43%
 INFO: Repository started (repository=...)
       "renovateVersion": "37.280.0"

 INFO: Repository has invalid config (repository=...)
       "error": {
         "validationSource": "renovate.json",
         "validationError": "Duplicate keys in JSON",
         "validationMessage": "\"Syntax error: duplicated keys \\\"prConcurrentLimit\\\" near imit\\\": 0,\\n\"",
         "message": "config-validation",
         "stack": "Error: config-validation\n    at checkForRepoConfigError (/home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/workers/repository/init/merge.ts:158:17)\n    at mergeRenovateConfig (/home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/workers/repository/init/merge.ts:182:3)\n    at getRepoConfig (/home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/workers/repository/init/config.ts:14:12)\n    at initRepo (/home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/workers/repository/init/index.ts:55:12)\n    at Object.renovateRepository (/home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/workers/repository/index.ts:62:14)\n    at attributes.repository (/home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/workers/global/index.ts:217:11)\n    at start (/home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/workers/global/index.ts:202:7)\n    at /home/vsts/.npm/_npx/59c0475bfd22776c/node_modules/renovate/lib/renovate.ts:18:22"
       }

 WARN: ensureIssue() is not implemented (repository=...)

 INFO: Repository finished (repository=...)
       "cloned": true,
       "durationMs": 2128

 INFO: Renovate was run at log level "info". Set LOG_LEVEL=debug in environment variables to see extended debug logs.

##[debug]Exit code 0 received from tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]Exit code 0 received from tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]STDIO streams have closed for tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]STDIO streams have closed for tool '/opt/hostedtoolcache/node/20.12.0/x64/bin/npx'
##[debug]Renovate done
##[debug]task result: Succeeded
##[debug]Processed: ##vso[task.complete result=Succeeded;]
Finishing: Run Renovate

I'm also a bit surprised about the double debug logs when validating the config. But I assume that's something behind the scenes that we can not control?

2024-04-08T13:55:33.9777685Z ##[debug]/opt/hostedtoolcache/node/18.20.0/x64/bin/npx arg: --yes --package renovate@latest -- renovate-config-validator
2024-04-08T13:55:33.9783363Z ##[debug]/opt/hostedtoolcache/node/18.20.0/x64/bin/npx arg: --yes --package renovate@latest -- renovate-config-validator
2024-04-08T13:55:33.9785474Z ##[debug]exec tool: /opt/hostedtoolcache/node/18.20.0/x64/bin/npx
2024-04-08T13:55:33.9785950Z ##[debug]exec tool: /opt/hostedtoolcache/node/18.20.0/x64/bin/npx
2024-04-08T13:55:33.9787305Z ##[debug]arguments:
2024-04-08T13:55:33.9787664Z ##[debug]arguments:
2024-04-08T13:55:33.9788002Z ##[debug]   --yes
2024-04-08T13:55:33.9798419Z ##[debug]   --yes
2024-04-08T13:55:33.9799776Z ##[debug]   --package
2024-04-08T13:55:33.9801713Z ##[debug]   --package
2024-04-08T13:55:33.9802376Z ##[debug]   renovate@latest
2024-04-08T13:55:33.9803064Z ##[debug]   renovate@latest
2024-04-08T13:55:33.9803409Z ##[debug]   --
2024-04-08T13:55:33.9805205Z ##[debug]   --
2024-04-08T13:55:33.9805550Z ##[debug]   renovate-config-validator
2024-04-08T13:55:33.9805926Z ##[debug]   renovate-config-validator
jycouet commented 3 months ago

I read INFO: Config validated successfully after npx --yes --package renovate@latest -- renovate-config-validator. Strange no?

KaVeKa commented 3 months ago

Yes indeed.. if I really mess things up like configuring 'true' where an integer is expected, the renovate validation fails as expected.

jycouet commented 3 months ago

I'll wait @Polymeta to confirm as well by closing the issue. Thx for quick exchange.

Polymeta commented 3 months ago

Hey all, thanks for the quick turnaround!

I can confirm that renovate now successfully runs again after the config got validated