Future Python 3.7+ Compatiblity, Interest, Support ?

[riker1]

(Earl) Zope has matured and can now safely handle the new world of Python 3.

The now-COVID19 universe has created a scenario/need/use-case/requirement of data encryption at all stages of the transaction process. My work involves the world of higher-ed. We're using Plone + (encryptingstorage) and my colleague's custom addon as a DMS.

Plone 5.2.1 + encryption storage has essentially been a killer-app for us. Our central IS office is struggling to implement something in FileNet and Oracle (yikes!). We've already got uptake, usage, and people working remotely and able to serve their constituencies.

--

Encryptingstorage is working very well for the most part but my concerns are as Py3K ages and coverage for the older versions fade (3.6 for example) are surprises going to occur? A lot has changed since the last 3 years a commit has been made.

--

I see that there are many components/dependencies that go into this storage implementation. They were tested (3+ years ago) on 3.4,3.5, and 3.6 (kmi, zlibstorage, webob, zope.component, etc). I'm using 3.6.9 on Ubuntu LTS 18.0.4.4 till that expires.

• I'm not a programmer per se, are the codebases for the dependencies and core components worth investing the time in making them further compatible?

• Is there anything that depends on that pesky zope.tempstorage for example? Old problems/new problems? Does moving to waitress create issues?

• I think we've been having an issue with plone.namedfile in our dexterity add-on. As a storage wrapper rather than a different storage implementation, should this matter?

• Is there any interest in seeing what breaks/works etc with newer python?

• IMHO, I think that Because of the move to Waitress in Plone 5.2.x, you seem to not have to worry about specifying webob, zope.component, or paster versions which is great.

I think encryptingstorage is a very good story for both Zope and especially Plone. I'd like to help make this a better-known feature etc.

[jugmac00]

Just a couple of random thoughts of a fellow Zope user:

• it is good that you make up your mind now, instead of at the eol of Ubuntu 18.04 or Python 3.6
• I'd validate whether Ubuntu supports Python 3.6 longer than its official eol on 2021-12-23
• there seems to be low action for cipher.encryptingstorage and its related packages, but at least they run on Python 3
• adding support for Python 3.7, 3.8 ... usually just means add those interpreter versions to the tox.ini for local tests and to .travis.yml for continuous integration tests, and then fix the usual warnings or sometimes errors (sometimes it's more work)
• here, we have no tox.ini so somebody has to create one
• the Travis (continuous integration) job ran the last time in Dec 2017 ( https://travis-ci.org/github/zopefoundation/cipher.encryptingstorage/builds ) - so it could be broken as of now (maybe there was an updated dependency or a transitive dependency)
• one of the maintainers (e.g. @icemac or @mgedmin or @strichter ) should change the Travis setting so it runs weekly
• once the tests are running again for Python 3.6 and you have a tox.ini, it is "easy" to run the tests also for Python 3.7, 3.8 and maybe for a pre-version of Python 3.9 - and then you see what needs to be fixed

[jugmac00]

I'm not a programmer per se, are the codebases for the dependencies and core components worth investing the time in making them further compatible?

If you rely on this package and thus on the dependencies, only you can say whether it is worth investing the time. Sometimes it is hard to tell in advance how much effort you'd have to invest. So think about the alternatives.

Is there anything that depends on that pesky zope.tempstorage for example? Old problems/new problems? Does moving to waitress create issues?

Can't say anything about tempstorage, but I do not understand the waitress thing. When I understand you correctly, you are running your app on Python 3 already, so there is no more ZServer, so you already use waitress or another wsgi server. So there is no more move.

Is there any interest in seeing what breaks/works etc with newer python?

Well, obviously there is interest for those using this package - and currently it looks like there are not many users. Instead of reporting a breakage or a warning from your local environment, I guess the way to go is as I outlined in the last comment - create a tox.ini and try to add support for newer versions.

I think encryptingstorage is a very good story for both Zope and especially Plone. I'd like to help make this a better-known feature etc.

I would love to read a blog article or in general more blog articles about best practices and use cases.

[d-maurer]

...

• Is there anything that depends on that pesky zope.tempstorage for example? Old problems/new problems? Does moving to waitress create issues?

tempstorage is typically used to store Zope sessions. There are long standing problems with it (aggravated with new ZODB releases). Those problems causes Plone a long time ago to replace its use of Zope sessions (by cookies). Thus, a stock Plone (wthout extensions) should not be affected.

Typical users will have no problems with waitress. Users which used special ZServer functionality (e.g. its protocal support for FTP) are affected (but someone is working on a ZServer reimplementation based on modern Python 3 libraries (asyncio)). In addition, some (advanced) functionality has been moved out of Zope and dumped into ZServer (even though it does not belong there) such as signal handling (and formerly WebDAV support). Likely, you will not be affected; and if affected, then the git repository allows to resurrect the code.

...

• IMHO, I think that Because of the move to Waitress in Plone 5.2.x, you seem to not have to worry about specifying webob, zope.component, or paster versions which is great.

I do not see any reason why the move ZServer -> waitress should affect this.

[jugmac00]

I set up a quick and dirty tox.ini locally, and yes, there seems that some work to be done, at least for Python 3.8 +.

  py36: commands succeeded
  py37: commands succeeded
ERROR:   py38: commands failed

Tests on Python 3.8:

Total: 564 tests, 3 failures, 0 errors and 0 skipped in 1 minutes 58.470 seconds.

Who migrated your application from Python 2.7 to Python 3.6? Or did you create this application on Python 3 directly?

[icemac]

I tried to activate weekly builds on TravisCI but I failed. To find out if feature is currently broken on TravisCI I created https://travis-ci.community/t/adding-cron-jobs-currently-broken/8147.

[mgedmin]

I added a weekly build in https://travis-ci.org/github/zopefoundation/cipher.encryptingstorage/settings:

It is building now.

FWIW if anyone wants to use cipher.encryptingstorage in production, I would recommend that you hire a cryptographer to audit it first (and keas.kmi, especially keas.kmi).

[riker1]

Who migrated your application from Python 2.7 to Python 3.6? Or did you create this application on Python 3 directly?

I gave up on Py2.7.x as soon as Mr. Bauer and his army made Zope and Plone Py3 ready. It also helped that we used Future and 2to3 as well to migrate our existing add-ons.

Must plug my humble boss, colleague, and friend who's been familiar with Zope since pure CMF days and also note that PloneCLI + Dexterity has really made content types much easier to design.

[icemac]

@riker1 Is this issue still relevant or can it be closed?

[riker1]

You can close.

I have been able to use the package with the latest Python 3.8.11 + Plone 5.2.10, & Volto.

[icemac]

@riker1 Please see the discussion in https://github.com/zopefoundation/meta/issues/193. Do you currently actively use this package, so we should keep maintaining it?

[dataflake]

@icemac The question is misleading. "we" is undefined. I am sure you're not offering perpetual support for this specific package, correct?

[icemac]

Sorry for not being precise enough. "We" is the Zope community, especially the users of this repository. I see my personal role as a release manager: cutting releases, trying out new versions of dependencies + fixing small problems occurring during the process and managing the zopefoundation projects. This also includes archiving projects which are no longer relevant for the community.

So the question is actually: Can this repository be archived because it is no longer used?

[riker1]

@icemac & @dataflake apologies for taking forever to reply. I can't speak for others (are there any?) in the community, but we're migrating off of this in favor of using encryption possibilities available within PostgreSQL using RelStorage.

[icemac]

@riker1 Thank you for your reply. I am going to archive this repository in the next couple of days. (Archiving is not permanent, it can be undone if someone wants to work on this repository again, currently I see no broader interest.)