search

zopefoundation / meta

Meta issues concerning many/all of the zopefoundation repositories.
Other
7 stars 6 forks source link

Future of the zope.app.* and z3c.* packages #193

Open icemac opened 1 year ago

icemac commented 1 year ago

During making GHA green again by dropping support for Python < 3.7 a came along many zope.app.* packages like:

Some of them were used in the ZMI of Zope 3 or at least contain browser views for it. Most of these packages had no release in the last 5 years.

Is someone still using these (kind of) packages? Could we archive a few of them to reduce the maintenance burden? (If they are actually still used and need changes, we could unarchive them, of course.)

dataflake commented 1 year ago

I am all for archiving those packages. As far as I know it's not a problem un-archiving packages if someone cares enough.

dataflake commented 1 year ago

I would extend this question to many z3c packages. I see you're working on those right now. How do you determine which are still in use so you don't waste your time?

dataflake commented 1 year ago

P.S.: I'd like to generalize that. Zope 3 the application server - roughly represented by the zope.app packages, has been dead for probably 15 years and those can be retired. Then there's the pile of packages we call the "Zope Toolkit" and I would claim that probably 70-80% of those are unused at this point.

My question still stands: How do you select packages to work on? Do you have a list of dependencies for your own projects in the back of your mind or are you just going through page after page of package listings inside this GH organization, most of which is probably dead?

jugmac00 commented 1 year ago

We use at least https://github.com/zopefoundation/zope.app.http on Python 3.5 (still supported by the Ubuntu security team), so please do not archive that one. And yes, we do plan to upgrade.

We also use a couple of z3c packages, so please announce a possible drop of support, although as mentioned, you could always unarchive them.

dataflake commented 1 year ago

Keep in mind that archiving does not remove existing releases, so if a user has an old application they can run and recreate that forever, regardless of archiving status. Unarchiving is only needed if someone is interested in continuing development and new releases. IMHO it's OK to then ask those people to take over that work as well.

I would consider archiving more of a marker for the few people who are still active that this is a package we don't have to worry about when it comes to mass-updates like the current retirement of EOL Python versions.

dataflake commented 1 year ago

P.S.: And no, I don't consider it feasible to announce every archival. Nothing will break for those who care for such a package. If there is an audience interested in further development they will notice the package status easily enough and are free to ask for a change of status - or reconsider and switch to a better-supported package, which is probably a wise choice.

jugmac00 commented 1 year ago

And no, I don't consider it feasible to announce every archival.

@icemac announced the archival of the zope.app. packages in bulk, so I assumed it would be possible to do the same for the z3c. packages.

I think the introduction of the meta tool also introduced a paradigm shift.

In the following I refer only to non-mainstream packages, ie not used by Zope core, or not so common packages.

Before the tool was introduced, we (as in any of the Zope contributors) just updated the packages we needed, or when somebody created an issue that a new release or new python support is necessary. So this was a community effort. Thousands of users, hundreds of contributors (or dozens at least).

Since the introduction of the meta tool, suddenly there was an attempt to keep each package up to date. And I also noticed that most of the work is now done by @icemac - either intentionally, or at least at the beginning there was some hesitation as the meta tool was not super easy to use. Meanwhile it is at least used by a couple of contributors.

And now @icemac speaks of a "maintenance burden" which needs to be reduced.

I certainly get that this is a burden, but imho this is a self-inflicted burden.

Why not just stop updating the repositories any more instead of archiving them? Then at least any user could work on them - without the step to ask any of the admins to reactivate it.

Don't get me wrong, I am very, very thankful what @icemac and the other maintainers do, but imho nobody asked to keep every package up to date.

icemac commented 1 year ago

I get a mail if GHA breaks for a package, so I decided to fix these packages. I vastly underestimated the effort necessary but now I've nearly done it.

I do not like the idea of just leaving the repositories as they are. This generates a false impression of them being active and it creates even more maintenance mess. I think the active packages should kept active the no longer used ones may rest in peace.

My current idea is to ensure that the tests run successfully for all currently active packages, create releases and then archive as much as possible in the namespaces zope.app.*, z3c.* and the more exotic ones.

But I'd like to gather some more opinions:

I think we should mark the repositories which are still in active use in projects int the wild, so they do not get archived. I did this for https://github.com/zopefoundation/Zope using a GitHub Topic and will do so for the repositories used in the projects I am maintaining.

dataflake commented 1 year ago

I fully agree with @icemac, if a package is no longer under active development then that fact must be communicated clearly in order to prevent bad assumptions and expectations. Archiving such a package is a good way to do that.

The GitHub topic is a similar marker and a good idea, but IMHO if we archive packages it would just repeat the same information that archived vs. unarchived status already provides.

An extensive cleanup among the 300+ packages in the zopefoundation organization is long overdue. My guess is that about half of them are dead or in vegetative state.

@jugmac00 you seem to promote some kind of "free for all" vision of an unmaintained garden that is allowed to just grow wild, a place where no one takes any responsibility for anything. I don't think that makes these projects any more attractive and I highly doubt it will encourage anyone to contribute. @icemac has been doing the opposite, he has taken responsibility, and he wants to manage that responsibility. Now you're saying "stop complaining, this is all your own fault"? Come on.

jugmac00 commented 1 year ago

FWIW, as a regular contributor I do not have permissions to create tags in the zope repositories.

Do you want me / all users to create an issue in each repository that you need to create a "maintained" tag? What is the consequence then? Will you maintain these repositories? Will you pass pypi and gh permissions to the one who created the issue?

I think it is good that we have a discussion as by far not everything is clear.

dataflake commented 1 year ago

I think we should mark the repositories which are still in active use in projects int the wild, so they do not get archived. I did this for https://github.com/zopefoundation/Zope using a GitHub Topic and will do so for the repositories used in the projects I am maintaining.

We need to distinguish between "used" and "there's a real need for further maintenance". I am sure you could find all of these old packages used somewhere in some custom application, but that doesn't mean much.

icemac commented 1 year ago

I am done with marking the packages I require. (Note: I did not mark the packages in the Zope 5 stack.) I have no problem to additionally maintain more packages which are used in projects of other people.

@jugmac00 Creating an issue for the packages you'd like to take care of sounds like a good idea. With reference to #191 package maintainers could get additional permissions in the repositories they take care of to circumvent problems during releases.

icemac commented 1 year ago

I am sure you could find all of these old packages used somewhere in some custom application, but that doesn't mean much.

If no-one speaks up for a package it can get archived for now until we find out that is needs to be maintained.

dataflake commented 1 year ago

@jugmac00 Creating an issue for the packages you'd like to take care of sounds like a good idea. With reference to #191 package maintainers could get additional permissions in the repositories they take care of to circumvent problems during releases.

Combining this with your previous comment that you don't have an issue maintaining other packages you don't use yourself I am reading this in answer to @jugmac's question what consequences the "maintained" tag has: Marking a repository as "maintained" does not mean that other contributors like @icemac will automatically take it over. He may voluntarily help, but there is no obligation. The onus is on the person who applies the tag to find such maintainers.

mauritsvanrees commented 1 year ago

The only zope.app package used in Plone 6 is zope.app.locales. For z3c:

z3c.caching
z3c.form
z3c.formwidget.query
z3c.objpath
z3c.pt
z3c.relationfield
z3c.zcmlhook

zope packages, other than zope.app.locales:

zope.annotation
zope.app.locales
zope.browser
zope.browsermenu
zope.browserpage
zope.browserresource
zope.cachedescriptors
zope.component
zope.componentvocabulary
zope.configuration
zope.container
zope.contentprovider
zope.contenttype
zope.copy
zope.datetime
zope.deferredimport
zope.deprecation
zope.dottedname
zope.event
zope.exceptions
zope.filerepresentation
zope.globalrequest
zope.hookable
zope.i18n
zope.i18nmessageid
zope.interface
zope.intid
zope.keyreference
zope.lifecycleevent
zope.location
zope.pagetemplate
zope.processlifetime
zope.proxy
zope.ptresource
zope.publisher
zope.ramcache
zope.schema
zope.security
zope.sendmail
zope.sequencesort
zope.site
zope.size
zope.structuredtext
zope.tal
zope.tales
zope.testbrowser
zope.testing
zope.traversing
zope.viewlet

In the Plone core development buildout we have a sources.cfg that extends the Zope sources.cfg as well, and on top the following repositories from the zopefoundation:

five.customerize
five.pt
Products.CMFCore
Products.CMFUid
Products.DCWorkflow
Products.ExternalEditor
Products.ExternalMethod
Products.GenericSetup
Products.PluggableAuthService
Products.PluginRegistry
Products.ZopeVersionControl
z3c.batching
z3c.caching
z3c.form
z3c.formwidget.query
z3c.relationfield
zodbupdate
Zope
icemac commented 1 year ago

@mauritsvanrees Thank you for the list. I added the maintained topic to all those packages (this way we also seem to have the Zope 5 stack included.)

There was one exception: https://github.com/zopefoundation/five.pt is already archived as it is deprecated since Zope 4.0a2. So it should be dropped (CC: @jensens).

icemac commented 1 year ago

@d-maurer @navytux @gforcada It would be helpful to know which additional zopefoundation packages you like to be kept alive.

mauritsvanrees commented 1 year ago

You are right about five.pt. I have removed it from the sources list now. Thanks for adding the tags.

dataflake commented 1 year ago

I'd add the following, no claim this is complete:

Products.BTreeFolder2
Products.MIMETools
Products.MailHost
Products.PythonScripts
Products.SQLAlchemyDA
Products.Sessions
Products.SiteErrorLog
Products.StandardCacheManagers
Products.TemporaryFolder
Products.ZCatalog
Products.ZMySQLDA
Products.ZODBMountPoint
Products.ZSQLMethods
ZEO
ZODB
icemac commented 1 year ago

@dataflake I added the packages you listed.

Currently we have 140 out of 281 active zopefoundation repositories having the maintained topic.

d-maurer commented 1 year ago

Michael Howitz wrote at 2023-2-9 04:02 -0800:

@d-maurer @navytux @gforcada It would be helpful to know which additional zopefoundation packages you like to be kept alive.

Beside the packages used by Plone, I am interested in z3c.unconfigure, and zope.formlib.

jugmac00 commented 1 year ago

We use the following packages, no claim this is complete:

z3c.pt
z3c.ptcompat -> needs a tag
zope.annotation
zope.app.http -> needs a tag
zope.app.publication
zope.app.publisher -> needs a tag
zope.authentication
zope.browser
zope.browsermenu
zope.browserpage
zope.browserresource
zope.cachedescriptors
zope.component
zope.componentvocabulary
zope.configuration
zope.container
zope.contentprovider
zope.contenttype
zope.datetime
zope.deprecation
zope.dottedname
zope.error
zope.event
zope.exceptions
zope.filerepresentation
zope.formlib
zope.hookable
zope.i18n
zope.i18nmessageid
zope.interface
zope.lifecycleevent
zope.location
zope.login
zope.pagetemplate
zope.password
zope.principalregistry
zope.processlifetime
zope.proxy
zope.ptresource
zope.publisher
zope.schema
zope.security
zope.securitypolicy
zope.sendmail
zope.server -> needs a tag
zope.size
zope.tal
zope.tales
zope.testbrowser
zope.testing
zope.testrunner
zope.traversing
zope.vocabularyregistry
navytux commented 1 year ago

@d-maurer @navytux @gforcada It would be helpful to know which additional zopefoundation packages you like to be kept alive.

@icemac, thanks for asking.

My personally focus is on ZODB stack:

Maybe @perrinjerome knows better about other packages.

perrinjerome commented 1 year ago

Except for what's already listed in this thread, I see we use:

in other words, I think we don't need anything else that what was already listed here.

Thanks !

mgedmin commented 1 year ago

I maintain zodbbrowser that depends on 

$ grep zope.app setup.py 
        "zope.app.pagetemplate",
        "zope.app.publication",
        "zope.app.authentication",
        "zope.app.component",
        "zope.app.server",
        "zope.app.session", # purely BBB for old Data.fs'es
        "zope.app.zcmlfiles",
            "zope.app.folder",
            "zope.app.container",
            "zope.app.testing",

(The last tree are dependencies for the test extra.)

The one remaining Zope 3 app in production (in long-term maintenance mode with no active development) depends on

$ grep zope.app setup.py 
        "zope.app.exception",
        "zope.app.securitypolicy",
        "zope.app.appsetup",
        "zope.app.authentication",
        "zope.app.basicskin",
        "zope.app.component",
        "zope.app.container",
        "zope.app.exception",
        "zope.app.file",
        "zope.app.folder",
        "zope.app.form",
        "zope.app.generations",
        "zope.app.pagetemplate",
        "zope.app.principalannotation",
        "zope.app.publication",
        "zope.app.publisher",
        "zope.app.schema",
        "zope.app.security",
        "zope.app.server",
        "zope.app.testing",
        "zope.app.zopeappgenerations",
        "zope.app.zcmlfiles",
        "zope.applicationcontrol",
        "zope.app.dublincore",
        "zope.app.catalog",
        "zope.app.intid",
        "zope.app.session",

I think I have sufficient permissions to unarchive a git repo if the need arises.

icemac commented 1 year ago

@mgedmin I added the maintained topic to the zodbbrowser dependencies.

icemac commented 1 year ago

@wosc Which packages do you need to be kept alive?

wosc commented 1 year ago

Thanks for thinking of us specificially! I feel honored. :) We do indeed still use "half of a Zope3" stack for our main production CMS, so I'd be very glad if the "core" of it could be kept alive, which I think mostly consists of

zope.app.appsetup
zope.app.pagetemplate
zope.app.publisher
zope.app.publication
zope.app.wsgi

From the z3c realm we're using z3c.pagelet + z3c.template quite heavily (via gocept.pagelet).

icemac commented 1 year ago

@wosc Thank you for sharing your list. I added the maintained topic to the z3c packages you mentioned. The zope.app ones already had this topic.