search

openedx / axim-engineering

GitHub Issue repository for the Axim engineering team
https://openedx.atlassian.net/wiki/spaces/COMM/pages/3241640370/Axim+Collaborative+Engineering+Team
4 stars 2 forks source link

NPM Ownership #23

Open sarina opened 2 years ago

sarina commented 2 years ago

There is an edx organization on NPM that currently publishes many packages. An organization can't be renamed but there is a process for moving packages between organizations. The edx organization or scope as NPM calls it, has both packages that are a part of the Open edX ecosystem as well as packages that are specific to edx.org. To reduce confusion we'll be moving the openedx packages to a new openedx scope in NPM.

Impact

Any downstream packages that depend on a package that has moved will need to update its dependencies and imports to pull the same files from the new package location and version.

Tasks

- [ ] https://github.com/openedx/axim-engineering/issues/806
- [ ] https://github.com/openedx/axim-engineering/issues/948

Tentative Plan

  1. Create a new openedx NPM org. (Done)
  2. Create a new openedx-npm-release-bot github and NPM users. (Done)
    1. Add the new user to the openedx NPM org and create some API tokens for it. (Done)
    2. Add the new github user to the openedx GitHub org and create a personal access token. (we use this user when semantic release has changes to commit back to the repo as part of the release.) (Done)
    3. Add OPENEDX_NPM_RELEASE_GITHUB_TOKEN and OPENEDX_NPM_RELEASE_NPM_TOKEN openedx GitHub org secrets. (Done)
  3. For each package that should move to the openedx org(List):
    1. Do a final publish of the package into the edx org if the latest code hasn't already been published.
    2. Update the code to scope the package to the openedx NPM org.
    3. Publish to the openedx org at the same version as in the edx org.
      • Outstanding Question: Will Semantic release let us do that? Might need to do a minor version bump.
    4. Update all projects in the openedx org that use this package to use the new openedx scoped one.
    5. Notify community of the change and newly available package so they can update any code they have.
  4. Remove the edx-semantic-release GitHub user from the openedx org.
  5. Transfer the credentials for the edx-semantic-release user back to the 2U SRE team so they can take full control of this user
  6. Ask 2U to remove Feanil from the edx NPM org.

Links

feanil commented 2 years ago

Note for when we're figuring this out: NPM recently released the ability to require 2FA for all members of an org. We should consider turning this on at the same time.

jmakowski1123 commented 2 years ago

Later H2

sarina commented 2 years ago

I would think @arbrandes should weigh in on this item's relative importance / size.

arbrandes commented 2 years ago

I think doing this makes a lot of sense I'll assign this to myself and do a little discovery on what kind of effort will be required. (Hopefully it'll be pretty easy to do.)

adamstankiewicz commented 1 year ago

Adding some related context from a recent issue:

However, we are also using the same NPM token for publishing over in the edX github organization (in frontend-component-footer-edx). As we rotated this secret in the openedx org, it's no longer valid in the edX org, causing our edX-specific NPM releases to fail.

I've escalated the issue of needing to rotate the NPM token for the edx github org to 2U SRE, but wanted to flag here that having distinct NPM tokens for the 2 Github organizations would be advantageous so there is no coupling between Open edX and 2U moving forward.

I'd also be happy to discuss the impacts of moving existing NPM packages to a new openedx specific org. My assumption in that case is that 2U would still rely on the existing edX NPM org once Open edX moves away from it.

feanil commented 1 year ago

@arbrandes please review this proposed migration plan for fixing the NPM ownership.

I've created the new openedx organization and am the owner of both that and the old edx organization. Thinking through the problem I came up with the following potential steps for how we might proceed with the migration.

Context

Proposed Plan

  1. Create a new openedx NPM org. (Done)
  2. Add the edx-semantic-release NPM user to the new openedx org as a member so they can publish any new packages they need to.
  3. For each package that should move to the openedx org(List TBD):
    1. Do a final publish of the package into the edx org if the latest code hasn't already been published.
    2. Update the code to scope the package to the openedx NPM org.
    3. Publish to the openedx org at the same version as in the edx org.
      • Outstanding Question: Will Semantic release let us do that? Might need to do a minor version bump.
    4. Update all projects in the openedx org that use this package to use the new openedx scoped one.
    5. Notify community of the change and newly available package so they can update any code they have.
  4. Replace the edx-semantic-release user in NPM and GitHub with an Open edX specific one.
    1. Create a new openedx-npm-release NPM and GitHub users.
    2. Add the new user to the openedx NPM org and create some API tokens for it.
    3. Add the new github user to the openedx GitHub org and create a personal access token. (we use this user when semantic release has changes to commit back to the repo as part of the release.)
    4. Update SEMANTIC_RELEASE_GITHUB_TOKEN and SEMANTIC_RELEASE_NPM_TOKEN openedx GitHub org secrets.
    5. Confirm that publishing in the openedx org still works.
    6. Remove the edx-semantic-release GitHub user from the openedx org.
    7. Remove the edx-semantic-release NPM user from the opnedx NPM org.
    8. Transfer the credentials for this user back to the 2U SRE team so they can take full control of this user
    9. Ask 2U to remove Feanil from the edx NPM org.
arbrandes commented 1 year ago

In general, looks good to me. The largest (in terms of work) will, of course, be 3.iv. Otherwise:

On 3. iii:

We're effectively renaming the packages, technically a breaking change that warrants major version bumps. (Or a restart to 1.0.0, which is what happened when react-intl was renamed to formatjs, but I think that would be a tad too extreme.) Doing so would also be a clear signal that this is the intended direction: as a user, you'll only get new versions under the new namespace. This also obviates the need to publish the same version under a new namespace.

Can't think of a downside except that it's not obvious just from comparing package names and versions that nothing changed except the name... which, again, would be confusing in an of itself, as a name change is a pretty big deal in the first place.

arbrandes commented 1 year ago

Just throwing this in here, as it may come in handy whatever we do:

https://openedx.atlassian.net/wiki/spaces/FEDX/pages/3755868171/How+to+backport+a+change+to+a+previous+version+with+semantic-release+for+JS+libraries

It contains instructions on how to manually trigger releases, which is something we might have to do.

feanil commented 1 year ago

So I think the next steps for this are to:

Mashal-m commented 8 months ago

@arbrandes @feanil, has the migration of @edx/brand to @openedx/brand been completed? I have found this reference ticket

feanil commented 8 months ago

@Mashal-m yes, the move of that particular package has already been completed. Since it was an alias already that one was a lot easier to move than most of the others.

arbrandes commented 8 months ago

Quoting @brian-smith-tcril from this Slack thread, this is the current plan:

  • Make a PR updating the paragon peer dep scope in frontend-platform, ensure it's marked as a breaking change
  • Make a PR moving the paragon dependency in frontend-component-header to a peer dep with the @openedx scope, ensure it's marked as a breaking change
  • Make a PR moving the paragon dependency in frontend-component-footer to a peer dep with the @openedx scope, ensure it's marked as a breaking change

Then if someone wants to use a new version of paragon in an MFE they will need to update those other dependencies, but they will be able to use paragon from the openedx scopeThis avoids supporting multiple scopes on new versions of packages with paragon as a peer dep.