Open mauritsvanrees opened 6 months ago
With some scripts and the GitHub CLI tool it might be easy to distribute the effort... we can create preparatory branches to switch, so the window of brokenness will not be that big...
One last item that might be a good idea to also add in this massive breakage would be to also rename master
to main
as default branches π€πΎ
I will have some days off next week (and without kids!) so I will try to work on it.
I already ported our internal packages at work to native namespaces, and before it bitrots I can re-apply the steps there ππΎ
@gforcada Trying things out, preparing, making tooling, that would be great. But for clarity: please do not yet make the actual move to native namespaces on any package. That is really for Plone 7, so next year.
I guess for a small namespace like borg
it would be fine in a Plone 6 minor version already: only borg.localrole
exists in this namespace as far as I know. I wanted to add plonetheme
there, but I see lots of plonetheme
packages on PyPI, so that would be for Plone 7 as well.
I already had a problem after moving zest.releaser
to native namespaces, because there is also a zest.pocompile
package that I usually install in the same venv or buildout, and this no longer worked until I moved that one over as well.
Indeed we could create a native
branch on each package, do the changes there, and on coredev have a PLIP config and on Jenkins a PLIP job to checkouts those branches, before doing anything on a coredev 7.0 branch.
Going from master
to main
on all Plone repos could be good, but I would make that a separate PLIP. It is something that can be tackled one repo at a time. But: if this makes tooling for the current PLIP easier, then sure, let's combine the work.
Sure, I was not planning on merging anything π but as you say, tooling needed some love as both zope.testrunner
and z3c.dependencychecker
were not capable of working with native namespaced distributions.
Fortunately that is fixed now, but there might be other tooling (zc.buildout
?) that might need some help as well, or a transition plan.
The idea of creating a main
branch with only those changes (src
layout and native namespaces) sounds good, specially if we tie it with periodic (monthly?) rebases and some CI (Jenkins) jobs that ensure things keep working.
Otherwise, if we wait until we want to open the door for Plone 7 changes, and things are not ready, nor tried already (on CI), we might face a stressful point where we stall development while waiting for tooling to get fixed.
See this funny issue in tox
: https://github.com/tox-dev/tox/issues/3247 I'm a bit surprised that I was the first one to notice and report it actually, given how widespread is tox
and collective.*
packages π
Anyway: there is a draft PR for borg.localrole
and I have a 7.0
branch on buildout.coredev
still to be pushed, that just updates borg.localrole
branch from master
to main
π₯ I will push it in the next few days.
Then we will need a Jenkins job to ensure it does work π€
@mauritsvanrees @gforcada In order to avoid the need for new major releases of all packages at the same time, could we do something like this to switch between namespace package implementations based on the Python major version?
(in __init__.py
)
import sys
if sys.version_info < (13,):
__import__("pkg_resources").declare_namespace(__name__)
Ah, no, of course we cannot, because native namespace packages require NOT having __init__.py
at all.
https://peps.python.org/pep-0420/#migrating-from-legacy-namespace-packages
That is for pkgutil
namespaces, which is yet another namespace implementation. This is not used in core Zope and Plone.
Not sure how much is worth it, but an idea came across:
If we keep this releases parallel to the production ones, we don't have to wait until Plone 7 is open, for then to notice that, oops X and Y tooling/packages/dependencies are broken.
I still need to add the jenkins job to run tests with the borg.localrole
PR I made some weeks ago... I'm a bit overwhelmed right now π΅βπ« π maybe this weekend I find some time π
Not sure how much is worth it, but an idea came across:
- create branches for all distributions of a given namespace with only the PEP 420 changes
- make alpha releases of them all
The problem with doing this now, is that this gets in the way of other changes that require a major version bump and that we want to include in Plone 6.x.
Take for example plone.app.event
:
pep-420
, do the changes, bump the version, and release 6.0.0a1. (Or you create branch 4.x for use in Plone 6.0 and 6.1, and bump the version on master.)zoneinfo
module from Python 3.9+ instead of pytz
. So they want to bump the version to 6.0.0a1 as well, but that is already taken by the pep-420
branch.If we imagine it really happens this way, it can still be fixed, but it will be confusing:
zoneinfo
changes, and then rerelease 6.0.0a1 as 7.0.0a1 with the pep-420 changes. Then within 6.0.0a* you would have one release with native namespaces and another with pkg_resources namespaces.zoneinfo
changes, and then 8.0.0a1 with the pep-420 changes.So: I think best not do this.
Crazy idea of my own:
pep-420
branch could be pushed I guess, so others can cooperate if further changes are needed./release/pep-420-test-do-not-use-in-production
. ;-) Then you can use that url as find-link.setup.py
in the script though.Sample PR of moving to a src-layout and the changes needed then: https://github.com/plone/plone.app.event/pull/384 I close that one because it is old and outdated, but as example it works.
Responsible Persons
Proposer: Maurits van Rees
Seconder:
Abstract
Move from
setuptools/pkg_resources
namespaces to native namespaces. This is a breaking change, so is for Plone 7.Motivation
Use of
pkg_resources
fromsetuptools
is deprecated. Native namespaces have been available since Python 3.3 and are now the recommended and most easy way to define namespaces.Assumptions
Products
namespace.pip
andsetuptools
and other packaging related software. Bugs may be less likely to get a fix.setuptools
. There are other ways to handle packaging and distribution nowadays.pkg_resources
namespaces within the same namespace, but this depends on the tools that you use, and whether you make a development checkout of a package. A combination that works today, might fail tomorrow with a newersetuptools
orpip
version. It is best to move over all packages from a single namespace at the same time.zope.*
packages have moved to non-native namespaces, allplone.*
packages could still use the oldpkg_resources
-style namespaces.Proposal & Implementation
plone
organisation that are used by thePlone
package should move to native namespaces. Others too where possible, and where we have control.collective
namespace.3.x
, not3.0.x
.__init__.py
from the namespace packages. InProducts.CMFPlone
we would removeProducts/__init__.py
.browser
,profiles
, andtests
directories. There are multiple ways to do this, but I propose we use asrc-layout
as this is supposed to make everything automatic. For example moveProducts
tosrc/Products
. See discussion here. This is optional, but seems best. It avoids problems like this when movingzest.releaser
over to using native namespaces. It will make some QA code easier, for example inplone/meta
, because all code will be insrc
, instead of in any of{src, plone, Products, etc}
.setup.py
: this PLIP is not about getting rid ofsetuptools
. We must then declare setuptools as build backend inpyproject.toml
, which we are already starting to do in Plone 6.Products
namespace. The pure Zope community can use help from the wider Plone community in this.Products
namespace.plone.*
, includingplone.app.*
.collective
,plonetheme
,borg
,zc
,z3c
,repoze
,five
.plonetheme.barceloneta
andborg.localrole
.Deliverables
collective
namespace.Risks
zope.testrunner
was already updated, andz3c.dependencychecker
as well.Participants
Links:
setup.py
deprecated? Short answer: no.pkg_resources
cross_pep420_pkg_resources
(orcross_pkg_resources_pep420
but that is basically duplicate data). The takeaway from that table, is that currently you can install a pep420 (native) and apkg_resources
package within the same namespace under some conditions: