Transition plan for moving Hyperledger Aries Cloud Agent Python to OpenWallet Foundation acapy

[swcurran]

Latest Update: 2024.10.09

This description will evolve as the transition of ACA-Py into the [OpenWallet Foundation] (OWF) and the transfer of the repo (now done) is made operational. Bookmark this issue to learn about the approach, the progress, the timeline, and guidance on what you should do in updating your deployment to reference the new home for the repository.

Goals/Decisions

• Names used in the repo move to OWF:
  • The new name of this repo is acapy — https://github.com/openwallet-foundation/acapy
    • A rediect is in place so if you use the old Hyperledger repo name, you get to the proper place.
  • The name of the project at OWF is "ACA-Py”, although ACAPY can be used as well. It is not an acronym, or if it is, it is “A Cloud Agent Python”, or in the GNU tradition “ACA-Py is a Cloud Agent Python"
  • The name of the source folder in the repo will be acapy_agent (the name "acapy" is already taken at PyPi).
  • Sub-projects of ACAPY will have repos prefixed by "acapy-", such as acapy-plugins (which has already been moved).
• The documentation site for ACA-Py continues to be the https://aca-py.org, although there might be a bump or two as we complete the post-move updates.
• The description in this issue will be the latest and greatest on the transition, and guidance on updating deployments to reference the new repo and artifacts.
• LTS requirements will be met by continuing to be able to publish 0.11 and 0.12 releases through to their respective end of life dates. That includes being able to push updates to PyPi and GHCR using the existing artifact names. Note that we have not yet declared an LTS 1.x version of the project.

Details

• [x] DONE - None needed. Research if there is a likelihood that we need to do a 0.11 and/or 0.12 LTS release. If so, do it from Hyperledger, before the move.
• [x] Update the README.md for the repo to announce the move and create a file acapy_to_owf.md that will contain updated information about the best practices in updating deployments to the new OWF home of the project/repository.
• [x] Complete release 1.0.1 that contains the current changes to main. This is the last Hyperledger-based release.
  • The update includes a deprecation notice in the README about the Hyperledger repo move to OWF.
  • It also contains a deprecation notices about the movement of the RFC 0160 connections, issue credential v1 and present proof v1 to the plugins repo, removing them from the core ACA-Py repo. That will be done in an upcoming release.
• [x] Before the move prepare PRs that implement the changes resulting from the move to OWF -- but hold off on merging them until after the move. Include documentation updates, GHA changes, and so on.
• [x] Move the repo to OWF.
• [ ] Merge the PRs prepared for the move to OWF.
• [ ] Add a token to the OWF repo that allows for the publishing of artifacts to the Hyperledger GHCR.
  • [ ] Update the GHCR publishing job (GHA) to publish release artifacts to BOTH the OWF and Hyperledger locations.
  • [ ] Add a new GHA to publish releases on the 0.11 and 0.12 branches (but not main) to the aries-cloudagent PyPi project.
• [ ] Produce a release from OWF (1.1 or 1.1.1) that publishes the artifacts to their new PyPi and old and new GHCR locations.
• Carry on...

[swcurran]

@WadeBarnes and @ryjones -- love to get your input on the plan. Same with new maintainers @thiagoromanos and @ff137, and any other deployers. Does this make it as easy as possible?

For those at the meeting today (@esune @jamshale @dbluhm @mepeltier @PatStLouis) -- note the subtle change I made to the repo that is to be kept at Hyperledger to be able to publish LTS releases. We'll create the fork immediately after the OWF move, but we'll only rename the repo to the current name when we have to publish an LTS release. That will allow the GitHub automatic redirect to work until that point -- which may be forever. 🤞

Feedback welcome!

[WadeBarnes]

• If and when we must do a first 0.11 and/or 0.12 LTS release, we will rename the repo to the old "aries-cloudagent-python" name. At that point the the automatic GitHub redirect to the OWF repo will be stopped. Ideally, we won't ever have to do that, or when we do, the word is out amongst deployers about the change.

I don't think a rename back to aries-cloudagent-python would ever be needed. The artifacts, their names, and where they are published is independent of the name of the repository.

[swcurran]

Awesome! We were thinking that it would be needed for the GHCR namespace. If not — all the better! I’ve updated the plan in the issue description.

[SeanBohan]

Couple of housekeeping issues:

• [ ] Mission Statement - LF is going to create a Project Charter for ACA-Py, I am getting that process started. We will a mission statement for the ACA-Py project to be included in the Charter. As this is moving over to the OWF, there is no need for a contribution agreement.

• [ ] As ACA-Py is coming into OWF as an Impact project, we will need a representative for the Technical Advisory Council (TAC). The TAC meets every other Weds and should be 2-6 hours per month committment.

• [ ] Does the ACA-Py community want to come up with their own logo or should I reach out to LF Marketing for help?

[swcurran]

Thanks, @SeanBohan.

Oh boy, a mission statement. :-). We’ll work on that. Can you direct me to some other Mission Statements of projects so we see the style of what is expected.

We have the bi-weekly ACA-Pug meeting tomorrow and will check in with the community on all three issues.

[jamshale]

One of the trickier things is the LTS. When we fork the openwallet-foundation repo back to hyperledger we'll need to change the name back to aries-cloudagent-python and adjust the github actions to point to hyperledger/aries-cloudagent-python again.

That's all good, but forking won't maintain any of the tags or releases. We'll need to manually create the tags and releases by commit id after forking. The existing images are still available after moving the repo. This is only to create new images and pypi packages.

All the existing github secrets will also be lost. They'll also need to be re-made after forking for publishing workflows.

We should make sure the forked repo no longer allows PR's except from a few users that will create patch releases.

[esune]

We'll need to manually create the tags and releases by commit id after forking. The existing images are still available after moving the repo. This is only to create new images and pypi packages.

Can pypi packages be pushed like it is possible with Docker images? If so, we could just manually push the relevant packages/images to the ghcr for the forked repo - this is only advisable if we expect the LTS releases to be replaced by new ones in the relatively near future and therefore we'd have to manually push only a limited amount of times.

We should make sure the forked repo no longer allows PR's except from a few users that will create patch releases.

+1 to this. Also issue creation (at least from external contributors) should be prevented and redirected to the new repo?

[jamshale]

You can manually publish both the pypi package and the ghrc.io images. For pypi you need the publishing account username/password and for ghrc.io you need a repo PAT with package:read,write,delete privileges.

Getting the github actions to work again and manually tagging the commits wouldn't be too hard with the correct repo rights either, but there is a few steps to do. Renaming the repo again after forking is critical for ghrc because that's how it decides the namespace.

[dbluhm]

I've produced images for a repo where the image name doesn't exactly match the name of the repo. I think the org name must match but the repo name is not required to be in the image name.

[jamshale]

Another thing to consider is changing the project and package name will change the references in plugins. For the managed plugin repo we can do this ourselves but in unmanaged plugins like traction innkeeper they will break if they install the new package without updating the source code.

[swcurran]

I’m hoping this is not an issue:

One of the trickier things is the LTS. When we fork the openwallet-foundation repo back to hyperledger we'll need to change the name back to aries-cloudagent-python and adjust the github actions to point to hyperledger/aries-cloudagent-python again.

The point of moving-and-forking-back before we merge the PRs that we’ve made for the move is that the forked version will not have changed from what was working in Hyperledger before the move. Is that not right? My hope is that the GHAs should still work, except for the repo name change from “aries-cloudagent-python” to “aries-cloudagent-python-lts”.

[jamshale]

I didn't describe this very well. The forking back will work yes if we do it before merging the PR's, but I'm pretty sure we lose the github secrets and the tags/releases after forking back.

The existing artifacts for the releases will still work via redirects but to create new LTS releases via github actions I'm pretty sure we need to recreate the pypi secrets and the tags.

[ryjones]

@jamshale correct. I will ask you all to reconsider any plan to fork it back to Hyperledger.

[swcurran]

We don’t need all the tags — we just need the two (or perhaps 5) we already have, and we know where they belong. Can we not add back or get new secrets? And the tags we need are on branches.

@ryjones — if we don’t have a hyperledger-based repo, how will we produce the needed PyPi and GHCR artifacts for LTS releases. We’re not doing the fork for fun — we think it is necessary.

[WadeBarnes]

We don’t need all the tags — we just need the two (or perhaps 5) we already have, and we know where they belong. Can we not add back or get new secrets? And the tags we need are on branches.

@ryjones — if we don’t have a hyperledger-based repo, how will we produce the needed PyPi and GHCR artifacts for LTS releases. We’re not doing the fork for fun — we think it is necessary.

It might be possible to publish the releases from the repo in OWF. The PyPi packages for sure. As far as the images in the GHCR are concerned, provided the correct permissions are used it might be possible to publish an image to the Hyperledger repos from and OWF repo. Have not tried, but technically it could work.

[WadeBarnes]

I think @ryjones has a point, we should try to do everything from the repo in OWF and then figure out what to do from there if we find something is not possible.

[swcurran]

Talking to @WadeBarnes and based on what @ryjones has said, here is what we now think is possible, without having to do a link:

• Since we have a branch in the repo for the LTS releases, we can still produce the PyPi artifacts from the branches. We'll figure out the best approach, but suggest that we update the current GHA to not produce a PyPi release for the 0.11/0.12 releases, and add a GHA that will produce the old named PyPi artifact for 0.11/0.12 releases.
• For the GHCR, we will adjust the current GHA to push all artifacts to both the Hyperledger GHCR with the old tags, and to the OWF GHCR using the new tags. If possible, we would not push the 0.11/0.12 artifacts to the OWF GHCR.
• I think the Docs publishing will scheme will just continue to work, but if not, we can always directly update the gh-pages branch.

Please review/confirm/adjust this and I'll update the description of this issue with the latest thinking.

[jamshale]

I think this is a good approach. It makes things a lot simpler. We could possibly generate the 0.11.lts and 0.12.lts artifacts for the new package and image names as well. Then someone on the hyperledger artifacts could move to the OWF artifacts without needing to upgrade.

As mentioned before. It is possible to do both the pypi publish and image uploads via command line for the old locations if someone has the correct account credentials. This really shouldn't happen very often.

[ryjones]

@jamshale For GHCR, it looks like we add a token (not using the runtime token) with access. https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry

I added ACAPY_PUB to OWF's secrets. the user is hyperledger-bot. You should be able to publish to Hyperledger's GHCR with that combination