[tracker] External dependencies that need python 3 porting

[gforcada]

If we ever want to move Plone to Python 3 (I guess we all want 😄), one of the very first tasks that we need to do is make sure that our external dependencies (i.e. python distributions that we rely on that are not Zope nor Plone controlled) are already python 2 and python 3 compatible.

Background For that during the Plone Conference 2016 in Boston I shamelessly copied the code from zope on python 3, (actual code and my fork), to Plone status on python 3.

The main difference between the two pages is that the Plone packages listing takes as an input the packages from buildout.coredev 5.1 test script (i.e. after running bin/buildout what's on the sys.path on bin/test.

With this list we are supposed to know the exact amount of packages that are needed for being able to run the Plone test suite.

Call to action As these dependencies are external, anyone can potentially work on them, and they can be worked in parallel.

Quick check list for contributors to this effort:

• update this story with a pull request/issue whenever you start working on it (or find an existing one)
• put your github id and the date you start working on it, so we can keep an overview of who is doing what and the chances of an effort being stalled
• review the usage of that dependency within Plone, maybe it can be dropped and thus we don't need that specific dependency to be ported?
• when the changes are merged, ask for a release on pypi!

Packages list Without further ado, the external dependencies that need porting.

• [x] mechanize
• [x] mockup
• [x] sourecodegen (only used by five.pt, see https://github.com/zopefoundation/five.pt/issues/8)
• [x] cssmin (abandoned only used on CMFPlone, see https://github.com/plone/Products.CMFPlone/issues/1800)
• [x] mailinglogger (version 5.0 supports py3)
• [x] pathtools
• [x] robotframework_ride (seems to be only required by plone.app.discussion, see https://github.com/plone/plone.app.discussion/pull/125)
• [x] robotframework_seleniumlibrary
• [x] z3c.caching
• [x] z3c.objpath
• [x] zope.broken (was be dropped, seems it was a dependency of zope.container, that was removed with upgrade zope.container)
• [x] Products.DateRecurringIndex (just released!! https://pypi.org/project/Products.DateRecurringIndex/)
• [ ] z3c.zcmlhook (being worked on @gforcada )
• [x] collective.xmltestreport (being worked on https://github.com/collective/collective.xmltestreport/pull/23 )
• [x] futures (py2 dependency)
• [x] functools32 (py2 dependency)
• [x] slimit (needs a release https://github.com/rspivak/slimit/issues/107 replaced by calmjs.parse #2616)
• [x] interlude (being removed https://github.com/plone/buildout.coredev/pull/517)
• [ ] collective.MockMailHost and Products.SecureMailHost (deps of plone.restapi, to be removed https://github.com/plone/plone.restapi/pull/623 )
• [x] zope.app.intid (being removed https://github.com/plone/plone.app.versioningbehavior/pull/41 )

[jamadden]

zope.broken is entirely obsolete. It was merged into ZODB back in 2010. Its current distribution is just a shim that imports ZODB.interfaces.IBroken. That known, classifiers aside, the BWC shim it provides should be Py3 compatible (though I would urge dropping any dependency on it).

I've recently been working on moving older/less maintained zope packages to Py3, and it was for those reasons I skipped zope.broken.

[hvelarde]

cssmin and slimit can be removed from the list if we drop the minifying feature from the core; I think we all must agree that this is not a feature we want to keep/maintain.

[jamadden]

I can’t speak definitvely, but seeing mechanize on python 3 also seems remote. The maintainers of zope.testbrowser have long since moved on to WebTest. I believe that was because no one wanted to move mechanize to python 3.

[datakurre]

@gforcada robotframework_ride should not a required dependency, and should probably be cleaned from our buildouts (it's probably only included in tests.cfg of buildout.coredev with plone.app.robotframework [ride]). It's an IDE for editing robot tests, but AFAIK core devs are fine with writing tests with their favourite text editors.

[datakurre]

The next release robotframework_selenium2library is about to support Python 3: https://groups.google.com/forum/m/#!topic/robotframework-users/McYQ81MakM8 https://github.com/robotframework/Selenium2Library/milestone/17

[datakurre]

futures just backport features from Python 3. It should be handled somehow conditionally.

[gforcada]

😮 thanks a lot to all for this quick feedback. Open mental note to everyone: again, if we get organized, things can start rolling, so please keep organizing!

[ale-rt]

I checked slimit:

• https://github.com/rspivak/slimit/blob/97d5645bf546e2d91dcc5709b6d2f1479c84c40b/CHANGES#L8
• https://github.com/rspivak/slimit/pull/96

[loechel]

https://github.com/plone/Products.CMFPlone/pull/2107 will remove cssmin as dependency

[metatoaster]

I don't think slimit is maintained anymore, as there have been no release for about three years now and there are a number of outstanding bugs, a couple of them will result in wrong code being generated when run through the minifier (others just simply slimit lexer breaking on ES5 keywords when it shouldn't). If Plone intends to keep js minification in core, I will have a replacement ready when the mangling capabilities are added to calmjs.parse which is being worked on. It will be a drop-in replacement.

[metatoaster]

Since issues related to slimit release for Python 3 is raised again (well rather the lack of a release, as per rspivak/slimit#65 and rspivak/slimit#102), I have a fork of Products.CMFPlone with this change to show that this change is basically a drop-in replacement using calmjs.parse - this also fixes a huge list of issues that breaks the minifier (including fixes for some script that would become malformed after running through slimit). I can turn that into a pull request if needed.

Unless this JS compression feature is to be removed in favor of whatever Node.js method that Plone is currently embracing, a decision should be made to accept a solution that does not result in malformed JS being generated (changelog for calmjs.parse lists all the fixes done).

[esteele]

@metatoaster Would you be willing to create a PR for that to get merged into CMFPlone? I'd hate for 5.2 to be held up over this one package.

[gforcada]

The other problematic package is z3c.zcmlhook is only in Pyrenees github org

[jensens]

this was all done and fixed