search

renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.66k stars 2.33k forks source link

Feature request: Rushjs monorepo support #3681

Open KevinGrandon opened 5 years ago

KevinGrandon commented 5 years ago

What would you like Renovate to be able to do? We have recently moved to a monorepo managed by Rushjs. Rush is similar to yarn workspaces, and projects are defined by entries in a top-level rush.json file. Currently it seems that there's some manual work to update rush lockfiles, and it would be great if renovate did this out of the box.

Describe the solution you'd like Enable renovate to work with rushjs monorepos. Perhaps custom lockfile logic could occur when detecting a project with a top-level rush.json file and a common/ folder.

Describe alternatives you've considered We might be able to handle lockfile updating ourselves, with a custom CI runner that builds and updates the lockfile.

Additional context We experienced this with the Fusion.js codebase after we migrated to a monorepo. We saw that renovate was opening up Pull Requests, but not updating the common lockfile.

rarkins commented 5 years ago

Are you able to describe the commands you use if you were to manually make the updates? Ie so I can try to replicate them with Renovate.

For lerna for example if we detect a package file is part of the monorepo then we make all changes and then run “lerna bootstrap” after installing the version of lerna specified

KevinGrandon commented 5 years ago

Hello. Sorry, here's a more complete example:

danwbyrne commented 4 years ago

I am in a similar situation at https://github.com/neo-one-suite/neo-one, we recently made an effort to migrate to rushjs. Renovate does all of the package.json updates appropriately but we need to be able to apply the changes from rush update --full to the PR or else they won't ever pass a CI check where we use rush install (not allowed to update the shrinkwrap file).

jlsjonas commented 4 years ago

@danwbyrne note that you actually just need rush update, which will reduce the amount of conflicts you get between dependency updates

@rarkins rush update after updating the package.json files will update the applicable lockfile (rush supports different package managers, which one is being used is being abstracted behind rush) In a CI environment you would use the following command to trigger rush node common/scripts/install-run-rush.js update.

The main caveat here, is the lockfile. This will cause a file conflict in every renovate PR as soon as one is merged; effectively being limited to 1 PR concurrency (in our testing using the above command as postUpgradeTask)

rarkins commented 4 years ago

@jlsjonas thanks for the info.

which one is being used is being abstracted behind rush

We would also need to detect/know that inside Renovate because we'd make sure we install the right manager and version prior to running rush.

In a CI environment you would use the following command to trigger rush node common/scripts/install-run-rush.js update.

Is this as an alternative to rush being globally installed or run via npx?

This will cause a file conflict in every renovate PR as soon as one is merged

I think "might" or "often" is more correct than "will"? i.e. if it's npm or yarn lockfiles then rush itself doesn't increase the chances of a conflict compared to the underlying manager, does it?

jlsjonas commented 4 years ago

We would also need to detect/know that inside Renovate because we'd make sure we install the right manager and version prior to running rush.

This can quite easily be detected based on the rush.json file, see https://rushjs.io/pages/maintainer/package_managers/#specifying-your-package-manager

Is this as an alternative to rush being globally installed or run via npx?

As an recommended alternative to being installed globally. npx is not an option as it would introduce a phantom node_modules folder (see https://rushjs.io/pages/maintainer/enabling_ci_builds/)

I think "might" or "often" is more correct than "will"? i.e. if it's npm or yarn lockfiles then rush itself doesn't increase the chances of a conflict compared to the underlying manager, does it?

It doesn't increase chances of conflicts indeed, however what we're experiencing (at least on Bitbucket Server/Stash) is that PR's already receive a conflict stamp (at least on the lockfile) if it's changed, even if there's no line-level conflicts. This might be an issue with BBS more than anything though.

rarkins commented 4 years ago

This can quite easily be detected based on the rush.json file

What's the fallback if none of the lines are uncommented? Or do you think it's a reasonable requirement that Renovate users must specify one?

BTW is it mandatory that rush projects include common/scripts/install-run-rush.js ?

This might be an issue with BBS more than anything though.

Sounds like it could be. Ideally it would only conflict if there are line-level conflicts.

jlsjonas commented 4 years ago

What's the fallback if none of the lines are uncommented? Or do you think it's a reasonable requirement that Renovate users must specify one?

From their getting started:

Choose a package manager: The template defaults to using PNPM, but you can also use NPM or Yarn. See NPM vs PNPM vs Yarn for guidance.

The rush.json file is generated as part of the rush init process, thus it will already contain one of the lines. Having neither should be considered an invalid configuration.

BTW is it mandatory that rush projects include common/scripts/install-run-rush.js ?

Again from their docs:

All Rush repos come with a script common/scripts/install-run-rush.js

So, yes.

Sounds like it could be. Ideally it would only conflict if there are line-level conflicts.

Definitely

mmkal commented 4 years ago

@rarkins was there any movement on this? Would the team be open to a PR adding basic rush support? I'd be happy to chip in, although I don't have a good sense of how big a lift it would be 🙃

My take on pnpm vs npm vs yarn with rush: I think it would be fine to only support rush_x_pnpm at least initially. The default rush setup is with pnpm, and it looks to be the rush development team's preferred package manager. So I suspect the majority of rush monorepos are using pnpm, and there'd be significant value in having those ones play nice with renovate. In any case, pnpm would make sense to tackle first.

Some more observations that may be relevant to a renovate-rush integration:

const rushConfig = RushConfiguration.loadFromDefaultLocation() rushConfig.projects.forEach(p => console.log(p.packageName, p.projectFolder)


- The `RushConfugration` class also has a method for finding the lock file: `rushConfig.getCommittedShrinkwrapFilename()`
- Editing any of the project's `package.json` files, then running `rush install` updates the lockfile automatically, which could make things easier for configurations other than those requesting lockfile-only changes.
- Rush and lerna aren't technically incompatible. The repo I've been testing rush on still has a `lerna.json` file, mostly for the ability to run `lerna exec` locally.

CC @octogonz who might correct me on some of the above points. Those are things I've noticed using rush on a public repo: https://github.com/mmkal/ts.

~One other thought: it may be possible to have a very bare-bones implementation which requires rush _and_ lerna. The leaf packages could be bumped in the established way (somehow avoiding lockfile changes), then `rush update` run at the end to update rush's lockfile.~ (removed - turns out renovate doesn't rely on lerna.json to discover projects)
rarkins commented 4 years ago

I would definitely be open to a PR, as we haven't moved this forward yet. Feel free to ping us here in this issue if/when you uncover any unforeseen decisions needing made.

The big challenge is that npm support is already a bit messy due to npm, yarn, pnpm and lerna all essentially being "package managers" for package.json files. It would be nice if we could classify them as separate managers internally but for now they all fall under lib/manager/npm/.

Would rush be like lerna in the sense that if we detect the presence of Rush then we use it instead of npm/pnpm/yarn?

Stuk commented 4 years ago

Having native support would be great!

I've set up Renovate with Rush+pnpm with a config similar to:

    "postUpgradeTasks": {
        "commands": [
            // ignore post-install scripts that might not work in the Renovate runner
            "echo 'ignore-scripts=true' >> common/config/rush/.npmrc",
            // Playwright caches browsers in the home directory, which doesn't work on Renovate
            "env PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 node common/scripts/install-run-rush.js update"
        ],
        "fileFilters": [
            "common/config/rush/browser-approved-packages.json",
            "common/config/rush/nonbrowser-approved-packages.json",
            "common/config/rush/repo-state.json",
            "common/config/rush/pnpm-lock.yaml"
        ]
    }

and it seems to work so far.

The ignore-scripts and Playwright config is specific to our setup, but I thought I'd document here in case it's useful to others.

mmkal commented 4 years ago

@Stuk this is great, thank you! I didn't know about postUpgradeTasks. Did you have to do anything special to allow renovate to find the package.json files? My assumption was that wouldn't work, since usually they're discovered by reading rush.json.

viceice commented 4 years ago

Renovate searches for all package.json files in the repo. We only filter some test folders with config:base preset

mmkal commented 4 years ago

@rarkins @viceice do postUpgradeTasks run on the renovate app, or only self-hosted? In the docs it says they only run when trustLevel is high, but that's a self-hosted configuration only.

I ask because I'm seeing renovate PRs getting errors along the lines of You need to run "rush update". And rush update is included in postUpgradeTasks. Example config: https://github.com/mmkal/ts/blob/main/renovate.json. Dashboard job logs - I can't find any logs indicating that the node common/scripts/install-run-rush update call was made: https://app.renovatebot.com/dashboard#github/mmkal/ts/235504626

rarkins commented 4 years ago

They don't run in the hosted app

octogonz commented 4 years ago

Renovate searches for all package.json files in the repo. We only filter some test folders with config:base preset

This is probably okay. It's not the Rush way, but it's probably close enough to be accurate.

My take on pnpm vs npm vs yarn with rush: I think it would be fine to only support rush_x_pnpm at least initially.

This is a reasonable bet to make. It might not matter too much however, since the rush update front end abstracts away many of the differences between package managers.

One other thought: it may be possible to have a very bare-bones implementation which requires rush and lerna.

This doesn't make sense to me. It would work for Lerna folder layouts that have been converted to Rush, but it probably would not work for a typical Rush folder layout.

Would rush be like lerna in the sense that if we detect the presence of Rush then we use it instead of npm/pnpm/yarn?

This is the main point. For a single repo, your algorithm might be like this:

  1. Edit package.json
  2. Run npm install
  3. Commit the result

Whereas for a Rush repo, invoking npm install or pnpm install is not going to work. Your bot MUST do it like this:

  1. Edit all the package.json files that you find
  2. Run ./common/scripts/install-run-rush.js update. This command has no dependencies and should work in pretty much any Node.js environment
  3. Commit the result, which may include other Rush config files that got adjusted
rarkins commented 4 years ago

In rush is there the concept of "internal" packages that Renovate should ignore by default? And if so then is it like Lerna and Yarn Workspaces where we should just ignore any internal package dependency where we find the same name in a package.json?

octogonz commented 4 years ago

Architecturally, the design is prescriptive: the Rush projects are enumerated explicitly in Rush's config files (mainly rush.json) and all other package.json files can be assumed to be test assets, or deadwood files from a previous build, or some other role that should probably be ignored. Rush never uses wildcards to discover projects because scanning folders is inefficient, plus a CI machine will often preserve leftover files from a previous build so that caches can be reused.

But this is a technical detail. Renovate would probably do fine to use a simple glob to find package.json files and a simple .gitignore type filter to allow users to manually exclude files that got processed incorrectly. It's not ideal, but it's a much simpler contract to support, particularly if you want to be compatible with many different versions of Rush.

octogonz commented 4 years ago

Ohhh wait a sec... I think you're actually asking about local project references (where the a dependency gets symlinked to another project in the repo, rather than being installed from NPM).

Yes, Rush does this. There are two models. The installation model we're moving towards is PNPM workspaces model, which use the workspace: specifier like in this example:

libraries/stream-collator/package.json

  . . .
  "devDependencies": {
    "@microsoft/rush-stack-compiler-3.5": "workspace:*",
    "@types/heft-jest": "1.0.1",
    "@types/node": "10.17.13",
    "@rushstack/eslint-config": "workspace:*",
    "@rushstack/heft": "workspace:*"
  }
  . . .

These are easy to detect and ignore. This model is opt-in as an experiment currently, and will become the default in Rush 6.

The longstanding older model used normal version ranges, like this example:

tsdoc-config/package.json

  "dependencies": {
    "@microsoft/tsdoc": "0.12.21",  // <--- reference to local project
    "ajv": "~6.12.3",
    "jju": "~1.4.0",
    "resolve": "~1.12.0"
  },

This is harder to detect. The dependency is considered a local reference (i.e. symlinked instead of installed) only if all of these things are true:

You can use the @microsoft/rush-lib API to answer these questions, but it's probably safe to skip this support, as the older model is going to be deprecated soon. The workspace: specifier is a superior design, and many repos are already opted-in to that mode.

rarkins commented 4 years ago

Thank you everyone for the interest and expert details on Rush.

File matching

We'll need to decide between the "traditional" npm manager approach of matching package.json and then looking for meta files like lerna.json or maybe a new approach of matching rush.json and then considering all the package.json files as "additional files" instead of "package files". The additional file approach seems cleaner although may need some improvements to our "autoReplace" logic first.

Manager conflicts

Assuming we define rush as a standalone "manager" in Renovate, we need to think what to do when Renovate matches a Rush project as both npm/yarn/pnpm (currently bundled into the npm manager) as well as with rush. It might work as-is (both managers update the same package.json files and don't really care if somehow they were already updated) but ideally we wouldn't do duplicate work or risk some type of conflict.

Obviously we could ask users to manually disable npm if you're using Rush, but normally we try to automate that type of logic away. Example logic we could use:

  1. If rush returns any files during the extraction phase then ignore/disregard any files that npm also returns, OR

  2. Dig into the rush matches to find all package.json files and ignore/disregard only those specific files for npm. If there happens to still be some non-rush package.json files then process them as-is with npm

Do you think (2) is a valid use case, or is (1) simplest and best?

Ignoring internal packages

Generally, if Renovate finds a dependency with a valid version then it tries to update it. For Lerna and Yarn Workspaces, we do a post-processing step to collect all package.json>name fields and then ignore any references to those packages in any other package files. We don't have any fancy logic to check if they're within the valid range or not, etc.

If Rush is moving towards the workspace: approach then that would be very simple because we would naturally ignore a non-semver reference anyway.

Artifact updating

Renovate uses the term "artifact updating" to mean any updating required after we've patched package files. e.g. may be lock files, checksums, vendored dependencies, or whatever Rush needs.

For most package managers we perform the artifact updating immediately after updating a dependency. i.e.

  1. Patch package file
  2. Run command to update artifacts

For current npm managers, we delay artifact updating until after all dependency updating / package file patching has been done. Which best describes Rush?

  1. Best to run the Rush command after each package file patch
  2. Need to wait until all patching is done
  3. Either works, but doing it once is most efficient

Remember that if we run the Rush command after each patch, it can mean that the same package in multiple package files can have inconsistent versions. e.g. if we want to update a dependency X from 1.0.0 to 1.0.1 in 5 different files then the first four times would mean both 1.0.0 and 1.0.1 are present while only after the 5th time would it be consistently 1.0.1.

Installing tools before updating artifacts

Do I understand correctly that there's no need to do something like npm install -g rush because the logic is self-contained in the committed ./common/scripts/install-run-rush.js? How about npm, yarn or pnpm - does Rush assume that they are installed globally or does it manage the installation of them too?

Commit file patterns

Can we have pre-defined patterns of which "rush files" to look for modifications for? Although the equivalent of git add --all might work most of time, we generally try to be explicit about which files we're committing just so we don't end up adding something we don't want to.

mmkal commented 4 years ago

My as-a-user, not-a-maintainer views (since I said I'd be willing to open a PR - that's looking ambitious now, but can try to help with hopefully accurate info):

  1. If rush returns any files during the extraction phase then ignore/disregard any files that npm also returns, OR
  2. Dig into the rush matches to find all package.json files and ignore/disregard only those specific files for npm. If there happens to still be some non-rush package.json files then process them as-is with npm

Do you think (2) is a valid use case, or is (1) simplest and best?

(1) looks simplest and best to me. I suspect the vast majority of rush monorepos just... use rush. If not, the first and best choice for those repos would be to fix that and not have secret projects that their monorepo manager isn't told about. The second choice would probably be to open an issue with rush asking them to fix whatever issue is preventing users from wanting to declare their packages in rush.json. The third option would be to create a new issue here saying "please support my hybrid rush use case" and see how many 👍 s it gets.

For current npm managers, we delay artifact updating until after all dependency updating / package file patching has been done. Which best describes Rush?

  1. Best to run the Rush command after each package file patch
  2. Need to wait until all patching is done
  3. Either works, but doing it once is most efficient

Remember that if we run the Rush command after each patch, it can mean that the same package in multiple package files can have inconsistent versions. e.g. if we want to update a dependency X from 1.0.0 to 1.0.1 in 5 different files then the first four times would mean both 1.0.0 and 1.0.1 are present while only after the 5th time would it be consistently 1.0.1.

For this one, it's pretty important that rush update is called once after all package patches have been applied. Rush has a setting ensureConsistentVersions which would error for repos that use it in the intermediate stages.

Do I understand correctly that there's no need to do something like npm install -g rush because the logic is self-contained in the committed ./common/scripts/install-run-rush.js? How about npm, yarn or pnpm - does Rush assume that they are installed globally or does it manage the installation of them too?

No need to npm install -g rush. But it looks like install-run-rush needs npm to exist globally.

Can we have pre-defined patterns of which "rush files" to look for modifications for? Although the equivalent of git add --all might work most of time, we generally try to be explicit about which files we're committing just so we don't end up adding something we don't want to.

In the repo I've been testing on it's common/config/rush/pnpm-lock.yaml. I suspect a fuller list would be common/config/rush/{pnpm-lock.yaml,shrinkwrap.yaml,yarn.lock,npm-shrinkwrap.json} based on a quick look at the rush code. I don't know if there are others though.

Again, take what I say with a pinch of salt, the above is just based on what I've seen as a (fairly recent) user.

octogonz commented 4 years ago

I agree with everything @mmkal said.

In the repo I've been testing on it's common/config/rush/pnpm-lock.yaml. I suspect a fuller list would be common/config/rush/{pnpm-lock.yaml,shrinkwrap.yaml,yarn.lock,npm-shrinkwrap.json} based on a quick look at the rush code. I don't know if there are others though.

It would be safe to match any file under common/config/*. Rush manages this folder very carefully and will report an error if an unrecognized file appears there.

Rush supports many other advanced version-related features that we could throw at you (e.g. autoinstallers, installation variants, allowedAlternativeVersions, preferredVersions, etc.). But 95% of your users won't use those features, or would be fine if Renovate Bot occasionally makes a PR that needs manual assistance to merge. Better to start simple and see what people actually ask about.

mscharley commented 3 years ago

@rarkins This has the help wanted tag but hasn't been updated in a few months. Is there anything outstanding that needs help specifically? ie, how can we help move this along? This would be very helpful for me to have at the moment.

rarkins commented 3 years ago

Next step is someone to summarise exactly what need to be done and how that fits into the Renovate general flow.

Eg is it a new manager, what external commands need running, etc.

mscharley commented 3 years ago

What would that look like? For reference, this is about one of the simplest possible rush repos out there: https://github.com/mscharley/node-presets

  1. Detect rush repository based on rush.json in the top level. My gut says that if this file exists it would replace any attempts at using npm/yarn/etc.
  2. Update package.json files as normal for NPM, either using the normal detection mechanisms or pulling the project list from rush.json: https://github.com/mscharley/node-presets/blob/master/rush.json#L350-L414
  3. Run node ./common/scripts/install-run-rush.js update to update lockfiles.
  4. You must make a commit at this point or the changelog won't work properly.
  5. Run node ./common/scripts/install-run-rush.js change --bulk --message "<arbitrary message here>" --bump-type <semver-type> to add changelog entries.
    • This could either tie into the existing commit message options for semanticCommits or perhaps something else
    • However, the bump type must tie into the fixed set of --bump-type {major,minor,patch,none} which are semver bump types but don't necessarily map to those existing configuration options.

How this fits internally into the Renovate lifecycle internally though I can't speak to, I've only used Renovate as a user. I just used this workflow now to bump TypeScript on the above repo and came up with the following PR for a TypeScript library bump:

https://github.com/mscharley/node-presets/pull/9

rarkins commented 3 years ago

So in that case:

Regarding the change command, are you sure that all users would want that run automatically, or might it need to be configurable? It looks similar to our existing bumpVersion option, which is opt-in.

mscharley commented 3 years ago

So, the change command adds these files into the repo: https://github.com/mscharley/node-presets/pull/9/commits/96c709219b10be2514ecc831f16ed03cfa0cc258

Other than adding cruft to a repo that doesn't use them, I don't believe they're actively harmful. A configuration option would be good. In lieu of adding a a configuration option, I would definitely run it. Adding it won't hurt people who aren't using the automatic changelog features but not having it will be a no-go for people who are as they would have to manually add those files to each PR, rendering the PRs very hard to use. If you do add it as a configuration option, it's safe to default to false as this is an opt-in feature from rush's side also.

bumpVersion would be analogous to controlling whether to run the update command, though probably shouldn't be configurable for a rush repo - not running update makes it impossible to then do an install anywhere till it is run as rush install will always honour the lock file or fail if it can't.

mscharley commented 3 years ago

Wait, I take that back after re-reading the config option, bumpVersion does sound appropriate for the change command and has the right options too, though an explicit "none" might be needed as well to say "Please include the changelog files, but don't actually increment the version".

mscharley commented 3 years ago

Add an updateArtifacts() function which runs install-run-rush.js update. I think we'd want to make sure it runs once for the entire rush project and not once per-package.json, although would it work if it ran multiple times in a row? (presumably with runs 2+ essentially doing nothing)

Multiple runs after adding all the updates should be idempotent and do nothing though obviously it's going to be a hit to runtime. I haven't tried, but running after each update will likely fail in many cases due to conflicting version specifications across the whole repository. rush has only one global resolution pool so if it can't resolve a single version that satisfies every package.json it's managing then it will fail to install/update.

yume-chan commented 3 years ago

This will cause a file conflict in every renovate PR as soon as one is merged

I think "might" or "often" is more correct than "will"? i.e. if it's npm or yarn lockfiles then rush itself doesn't increase the chances of a conflict compared to the underlying manager, does it?

rush init adds the following into .gitattributes to disable file level merge on lockfiles

https://github.com/microsoft/rushstack/blob/a13865bef9a20dab28c044be3504c7326bfe94b1/apps/rush-lib/assets/rush-init/%5Bdot%5Dgitattributes#L1-L6

# Don't allow people to merge changes to these generated files, because the result
# may be invalid.  You need to run "rush update" again.
pnpm-lock.yaml               merge=binary
shrinkwrap.yaml              merge=binary
npm-shrinkwrap.json          merge=binary
yarn.lock                    merge=binary

So multiple PRs will definitely cause conflicts.

yume-chan commented 3 years ago

How about npm, yarn or pnpm - does Rush assume that they are installed globally or does it manage the installation of them too?

Rush will install its own copy of npm, yarn or pnpm

https://github.com/microsoft/rushstack/blob/90290acbaea901875c1e983cacf6942c3ab04fec/apps/rush-lib/src/logic/installManager/InstallHelpers.ts#L70-L74

  /**
   * If the "(p)npm-local" symlink hasn't been set up yet, this creates it, installing the
   * specified (P)npm version in the user's home directory if needed.
   */
  public static async ensureLocalPackageManager(
F3n67u commented 3 years ago

@octogonz I've set up Renovate with Rush with a config like below:

  "postUpgradeTasks": {
    "commands": [
        "node common/scripts/install-run-rush.js update",
        "node common/scripts/install-run-rush.js change --bulk --message \"{{{prTitle}}}\" --bump-type patch"
    ],
    "fileFilters": [
        "common/config/rush/browser-approved-packages.json",
        "common/config/rush/nonbrowser-approved-packages.json",
        "common/config/rush/repo-state.json",
        "common/config/rush/pnpm-lock.yaml",
        "common/changes/**/*.json"
    ],
    "executionMode": "branch"
  }

pnpm-lock.yaml update correctly, but change json file cannot generate because rush change won't work when have uncommitted changes: pnpm-lock.yaml and package.json.

Rush Multi-Project Build Tool 5.49.0 - https://rushjs.io
Node.js version is 14.16.1 (LTS)

Starting "rush change"

The target branch is origin/master
No changes were detected to relevant packages on this branch. Nothing to do.

Warning: You have uncommitted changes, which do not trigger prompting for change descriptions.

is there any solution to this problem?

F3n67u commented 3 years ago

@octogonz I've set up Renovate with Rush with a config like below:

  "postUpgradeTasks": {
    "commands": [
        "node common/scripts/install-run-rush.js update",
        "node common/scripts/install-run-rush.js change --bulk --message \"{{{prTitle}}}\" --bump-type patch"
    ],
    "fileFilters": [
        "common/config/rush/browser-approved-packages.json",
        "common/config/rush/nonbrowser-approved-packages.json",
        "common/config/rush/repo-state.json",
        "common/config/rush/pnpm-lock.yaml",
        "common/changes/**/*.json"
    ],
    "executionMode": "branch"
  }

pnpm-lock.yaml update correctly, but change json file cannot generate because rush change won't work when have uncommitted changes: pnpm-lock.yaml and package.json.

Rush Multi-Project Build Tool 5.49.0 - https://rushjs.io
Node.js version is 14.16.1 (LTS)

Starting "rush change"

The target branch is origin/master
No changes were detected to relevant packages on this branch. Nothing to do.

Warning: You have uncommitted changes, which do not trigger prompting for change descriptions.

is there any solution to this problem?

@rarkins Another problem: why {{{prTitle}}} always empty in change command? I have enable allowPostUpgradeCommandTemplating

DEBUG: Executing command (repository=*****, branch=*****)
       "dep": "***** **** ****",
       "command": [
         "node common/scripts/install-run-rush.js change --bulk --message \"\" --bump-type patch"
       ]
rarkins commented 3 years ago

@F3n67u can you try using commitMessage instead of prTitle as that might have a better chance of working

renoirb commented 3 years ago

Issue I see in some projects is SOC compliance, making an equivalent of Renovate would require to create a branch with a PR with that change.

you would have to create a CommandLineParser and CommandLineAction that does Git things, probably also an HTTP client to create a PR.

That's what I will have to do.

omairvaiyani commented 2 years ago

@octogonz I've set up Renovate with Rush with a config like below:

  "postUpgradeTasks": {
    "commands": [
        "node common/scripts/install-run-rush.js update",
        "node common/scripts/install-run-rush.js change --bulk --message \"{{{prTitle}}}\" --bump-type patch"
    ],
    "fileFilters": [
        "common/config/rush/browser-approved-packages.json",
        "common/config/rush/nonbrowser-approved-packages.json",
        "common/config/rush/repo-state.json",
        "common/config/rush/pnpm-lock.yaml",
        "common/changes/**/*.json"
    ],
    "executionMode": "branch"
  }

Using this configuration we get a "Artifact update problem" with the following error:

Post-upgrade command 'node common/scripts/install-run-rush.js update' does not match allowed pattern '^pwd$'

Any ideas what this pattern refers to?

F3n67u commented 2 years ago

@octogonz I've set up Renovate with Rush with a config like below:

  "postUpgradeTasks": {
    "commands": [
        "node common/scripts/install-run-rush.js update",
        "node common/scripts/install-run-rush.js change --bulk --message \"{{{prTitle}}}\" --bump-type patch"
    ],
    "fileFilters": [
        "common/config/rush/browser-approved-packages.json",
        "common/config/rush/nonbrowser-approved-packages.json",
        "common/config/rush/repo-state.json",
        "common/config/rush/pnpm-lock.yaml",
        "common/changes/**/*.json"
    ],
    "executionMode": "branch"
  }

Using this configuration we get a "Artifact update problem" with the following error:

Post-upgrade command 'node common/scripts/install-run-rush.js update' does not match allowed pattern '^pwd$'

Any ideas what this pattern refers to?

@omairvaiyani you need add ＂node common/scripts/install-and-run-rush.js" to https://docs.renovatebot.com/self-hosted-configuration/#allowedpostupgradecommands

rarkins commented 2 years ago

You need to be running self-hosted.

KadoBOT commented 1 year ago

You need to be running self-hosted.

What's the status for non-self-hosted? I went thru a couple of comments, and I'm unsure if I can or not use Renovate with my rush mono repo.

rarkins commented 1 year ago

postUpgradeTasks aren't supported in the hosted. So you'd need to use e.g. the Renovate GitHub Action

lzm0x219 commented 1 year ago

Will it support？

wtuminski commented 1 year ago

This is a working config for self-hosted:

renovate.json:

"postUpgradeTasks": {
        "commands": [
            "git config --global user.email <some_email> && git config --global user.name <some_user-name>",
            "node common/scripts/install-run-rush.js update",
            "git add -A .",
            "git commit --no-verify -m 'some commit message'",
            "node common/scripts/install-run-rush.js change --bulk --message \"{{{prTitle}}}\" --bump-type patch",
            "git reset --soft HEAD~1"
        ],
        "fileFilters": [
            "common/config/rush/browser-approved-packages.json",
            "common/config/rush/nonbrowser-approved-packages.json",
            "common/config/rush/pnpm-lock.yaml",
            "common/changes/**/*.json"
        ],
        "executionMode": "branch"
    }

and in config.js:

    allowedPostUpgradeCommands: [
        '^node common/scripts/install-run-rush.js update',
        '^node common/scripts/install-run-rush.js change',
        '^git config',
        '^git add',
        '^git commit',
        '^git reset',
    ]

We should commit changes before rush change if we want Rush to produce change files. Git has to be configured as Renovate configures it after post-upgrade tasks.

rarkins commented 1 year ago

This is not the location for technical support. Please raise a discussion per question instead, and one at a time

rarkins commented 1 year ago

Your questions are unrelated to Rush specifically so please stop the noise

hhanesand commented 8 months ago

Thanks for posting that config wtuminski 🙏 Here's a tweaked version that handles mutations in .npmrc files when using hostRules, and doesn't create a commit (rush change only ignores unstaged changes).

renovate.json

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "postUpgradeTasks": {
    "commands": [
      "git config --global user.email <some_email> && git config --global user.name <some_user-name>",
      "node common/scripts/install-run-rush.js update --ignore-hooks 1>&2",
      "git add -A .",
      "git reset HEAD - '*.npmrc'",
      "node common/scripts/install-run-rush.js change --bulk --message \"{{{prTitle}}}\" --bump-type patch 1>&2"
    ],
    "fileFilters": [
      "common/config/rush/**/*",
      "common/changes/**/*.json"
    ],
    "executionMode": "branch"
  },
}

config.js:

"allowedPostUpgradeCommands": [
    "^node common/scripts/install-run-rush.js update",
    "^node common/scripts/install-run-rush.js change",
    "^git config",
    "^git add",
    "^git reset"
  ],