search

zopefoundation / persistent

automatic persistence for Python objects
https://pypi.org/project/persistent/
Other
46 stars 28 forks source link

Use Python 3.6.2 on MacOS #122

Closed mgedmin closed 4 years ago

mgedmin commented 4 years ago

Our Python 3.6 builds on MacOS started failing with a

Couldn't use data file '/Users/travis/build/zopefoundation/persistent/.coverage': Safety level may not be changed inside a transaction

error from coverage run, after the test suite is done and it's time to collate and store coverage data. This is probably caused by a new release of coverage.py (5.0.3; the last successfull built had coverage 4.5.4) that uses sqlite for .coverage and is reported upstream as https://github.com/nedbat/coveragepy/issues/703.

Other platforms and other Python versions do not have this problem. Other platforms use a newer Python 3.6.x dot release, while MacOS is pinned to 3.6.0.

This PR is an experiment to see if a different Python 3.6.x dot release helps. Turns out it does!

We have a few other zopefoundation packages using Python 3.6.1 for MacOS wheel building:

Whereas these are pinned to 3.6.0 (and I wonder if we're about to see similar failures):

I'd appreciate additional attention from @jamadden and @fgregg, since IIRC you are familiar with Mac OS subtleties. (Wasn't Python 3.6.1 the release that broke ABI compatibility with 3.6.0, and 3.6.2 again broke ABI compatibility with 3.6.1 to restore compatibility with 3.6.0? Should we use 3.6.2 for wheel building instead of 3.6.1?)

I don't use Macs myself, but I care about Travis builds on git master being green.

jamadden commented 4 years ago

Yes, 3.6.1 broke the ABI (for all platforms). I've confirmed the fix is present in 3.6.2 and later. So I wouldn't recommend pinning to 3.6.1 if we have something later available. (It's entirely possible none of our projects use the affected symbol, PySlice_GetIndicesEx, in which case it should be a non-issue.)

I've been trying to track down the sqlite3 or SQLite change that might be responsible for the coverage breakage and so far I've come up empty.

I haven't found anything obvious in the SQLite release notes going back quite a ways. Any idea if the version of SQLite that the two Python versions were compiled against changed?

As far as CPython goes, the diff between 3.6.0 and 3.6.1 is quite small and doesn't seem like it would be responsible, although it does change the exact circumstances under when new transactions are automatically begin in autocommit mode.

EDIT: We both identified the same commit in 3.6.1 as fixing the issue. The comments on the coverage issue are more clear than what I had offered here. Removed my large diff.

fgregg commented 4 years ago

let's pin it to 3.6.2 as @jamadden.

mgedmin commented 4 years ago

(It's entirely possible none of our projects use the affected symbol, PySlice_GetIndicesEx, in which case it should be a non-issue.

BTrees/BTrees/BTreeItemsTemplate.c calls PySlice_GetIndicesEx. BTrees is also one of the projects that are building Mac OS wheels on Python 3.6.1.

ripgrep finds no other calls to PySlice_GetIndicesEx in my ~/src/zopefoundation/*.

mgedmin commented 4 years ago

Pinned to 3.6.2. CI passes.