Coverage reporting breaks with Python 3.8 #1173

Open BuildStream-Migration-Bot opened 3 years ago

See original issue on GitLab In GitLab by [Gitlab user @cs-shadow] on Oct 17, 2019, 11:05

Job #323086149 failed for 6fbbad12f0d87b22dce7164b2c389f400ba5150d.

I haven't looked into details of why this is failing. Creating this ticket for awareness at least.

In GitLab by [Gitlab user @cs-shadow] on Oct 17, 2019, 11:29

mentioned in merge request !1647

In GitLab by [Gitlab user @cs-shadow] on Oct 17, 2019, 11:31

Also, maybe related, Python 3.8 tests for bst-plugins-container are also failing. The error seems to come from buildstream code but I'm not 100% sure at this time.

See https://gitlab.com/BuildStream/bst-plugins-container/issues/19 for details of that issue.

In GitLab by [Gitlab user @BenjaminSchubert] on Dec 12, 2019, 16:38

This is my findings around https://gitlab.com/BuildStream/buildstream/tree/bschubert/python38.

The problems in the tests seem to be related to the asyncio loop with coverage. I can confirm that as when removing the coverage and running tests with --no-cov, the tests do pass.

In GitLab by [Gitlab user @juergbi] on Dec 15, 2019, 17:41

mentioned in merge request !1771

In GitLab by [Gitlab user @tpollard] on Dec 16, 2019, 15:18

Done a little digging. Turning coverage off & bumping to a newer version of the 3.8 test image with the latest buildbox gives 4 recursion errors, which seem to come from shutils copy changes. Looks to be a recursion error in fast_copy, which is listed as a change to use it as a platform specific syscalls for efficiency:

Next I've added a hard timeout & tested against the full release of Coverage 5.0. The errors here are hit with the cython plugin. They've switched from json to sql for the datastore (however, can still output json) & have also changed when they write to storage, adding parallel = True to coverage conf as recommended doesn't seem to 'fix' the issue. As can be seen in the backtrace the error is stemming from their sql module. The cython plugin upstream hasn't updated since 4.0, so I presume we've got some incompatibility here.

In GitLab by [Gitlab user @tpollard] on Dec 20, 2019, 15:25

marked this issue as related to #1246

In GitLab by [Gitlab user @cs-shadow] on Dec 24, 2019, 08:50

Update on the recursion errors: I recently tried to reproduce them locally but couldn't. Turns out it was caused due to https://bugs.python.org/issue38688, which is now fixed. I was using a newer testsuite image which comes with Python 3.8.1, that has this patch. Whereas the old tests were run on Python 3.8.0.

I am currently testing this on CI as well. If that passes, I can tidy up that into a merge request without coverage enabled.

In GitLab by [Gitlab user @cs-shadow] on Dec 24, 2019, 10:10

mentioned in commit 75c201bc378c1296b369cb6f292e29de3f19c13d

In GitLab by [Gitlab user @cs-shadow] on Dec 24, 2019, 12:01

mentioned in commit 4986195bdaaa8c6ad9b562f4d52b2f041f738846

In GitLab by [Gitlab user @cs-shadow] on Dec 24, 2019, 16:41

mentioned in commit 807f275a6639d4596d050753d488537511df38fe

In GitLab by [Gitlab user @cs-shadow] on Dec 25, 2019, 13:14

mentioned in commit 598ebf33c1ed0c9a191a5e3a671c3f84330bfca2

In GitLab by [Gitlab user @cs-shadow] on Dec 25, 2019, 13:14

closed via merge request !1647

In GitLab by [Gitlab user @juergbi] on Dec 26, 2019, 07:41

Reopening as it's not working yet with coverage enabled.

In GitLab by [Gitlab user @juergbi] on Dec 26, 2019, 07:41

reopened

In GitLab by [Gitlab user @traveltissues] on Dec 31, 2019, 09:34

https://gitlab.com/BuildStream/buildstream/-/jobs/391442667 the examples seem to be failing

In GitLab by [Gitlab user @cs-shadow] on Jan 14, 2020, 23:05

changed title from {-Tests fail-} with Python 3.8 to {+Coverage reporting breaks+} with Python 3.8

In GitLab by [Gitlab user @cs-shadow] on Jan 14, 2020, 23:05

changed the description

In GitLab by [Gitlab user @cs-shadow] on Jan 15, 2020, 00:07

mentioned in commit 7e76e5c263617827336239ac65fe837d6b35ee95

In GitLab by [Gitlab user @cs-shadow] on Jan 15, 2020, 00:12

mentioned in merge request !1796

In GitLab by [Gitlab user @jjardon] on Apr 5, 2020, 11:22

mentioned in merge request !1856

In GitLab by [Gitlab user @jjardon] on Apr 5, 2020, 12:29

marked this issue as related to #1283

In GitLab by [Gitlab user @jjardon] on Apr 5, 2020, 12:29

removed the relation with #1283

In GitLab by [Gitlab user @jjardon] on Apr 5, 2020, 12:29

marked this issue as related to #1283

In GitLab by [Gitlab user @BenjaminSchubert] on May 8, 2020, 17:12

Coverage 5.1 seems to be better at handling python3.8 However, it breaks with our usage of cython.

We are currently hitting: https://github.com/nedbat/coveragepy/issues/972 if trying to run with coverage 5.1.

In GitLab by [Gitlab user @mzabaluev] on Jun 7, 2020, 10:13

Any resolution of the "no running event loop" issue? I'm getting hit with it on Fedora 32 with buildstream-1.4.1-2.fc32. The exact reported bug is actually #1246, but it was closed in favor of this non-specific one.

In GitLab by [Gitlab user @juergbi] on Jun 7, 2020, 10:25

General support for Python 3.8 was added in BuildStream 1.4.2, see !1830. The test coverage issue still exists, however, this shouldn't block normal use of BuildStream. I.e., the Fedora package needs to be updated. Please report if you encounter any Python 3.8 issues with BuildStream 1.4.2 or later that are not related to test coverage.

In GitLab by [Gitlab user @juergbi] on Aug 12, 2020, 12:35

mentioned in commit 30d7e9ace1a2d2c60b9cbdc96f6a177b9f81d71d

In GitLab by [Gitlab user @juergbi] on Aug 12, 2020, 12:35

mentioned in commit 62dbb18e9a75060dd07b30fd9aeeea1dcc7d4051

In GitLab by [Gitlab user @juergbi] on Aug 12, 2020, 16:32

mentioned in commit 4a6ef6a2bdec9c22c915d38de537c1ad1e17da51

In GitLab by [Gitlab user @juergbi] on Aug 12, 2020, 16:37

mentioned in commit e5afbd7eb2cac387ec772b27a57c03769c0f5fd6

In GitLab by [Gitlab user @tristanvb] on Nov 6, 2020, 09:56

removed the relation with #1283

apache / buildstream

Coverage reporting breaks with Python 3.8 #1173