Closed tisto closed 6 years ago
jone
commented
10 years ago Ohh, those tests..
We need to rewrite them, but there are also tests which extend the CMFEditions tests (test_IntegrationTests.py), which are still PloneTestCase tests :unamused:
jone
commented
10 years ago @tisto I've made a pull request which fixes the failing tests (plone/plone.app.versioningbehavior/pull/7). You mentioned testing isolation problems, what are the indications for those?
tisto
commented
10 years ago Thanks! Will give it a try as soon as the Plone 5 build is green again.
tisto
commented
10 years ago Update: Pull request has been merged. Putting p.a.versioningbehavior into the PloneTestCase alltests groups leads to test failures in plone.outputfilters. We can at least run the p.a.versioningbehavior tests now. Tests still needs to be migrated to plone.app.testing...
tisto
commented
10 years ago @jone Adding p.a.versioningbehavior to the plone_app_testing alltests group leads to 30 test failures in plone.caching and plone.app.caching:
Error in test test_contenttype_class_lookup_browser_view (plone.app.caching.tests.test_lookup.TestContentItemLookup)
Traceback (most recent call last):
File "/Users/timo/.buildout/eggs/unittest2-0.5.1-py2.7.egg/unittest2/case.py", line 340, in run
testMethod()
File "/Users/timo/.buildout/eggs/plone.app.caching-1.2.1-py2.7.egg/plone/app/caching/tests/test_lookup.py", line 151, in test_contenttype_class_lookup_browser_view
z3c.caching.registry.register(DummyContent, 'rule1')
File "/Users/timo/.buildout/eggs/z3c.caching-2.0a1-py2.7.egg/z3c/caching/registry.py", line 144, in register
return registry.register(obj, rule)
File "/Users/timo/.buildout/eggs/z3c.caching-2.0a1-py2.7.egg/z3c/caching/registry.py", line 64, in register
raise LookupError("Explicit mode set and ruleset %s not found" % rule)
LookupError: Explicit mode set and ruleset rule1 not foundAny idea how this could be related? There is always the possibility that plone.caching and plone.app.caching have test isolation issues.
jone
commented
10 years ago @tisto I'm currently on vacation for another week, I'll probably not be able to look in into it before. It's probably another isolation problem, probably caches which are not invalidated..
tisto
commented
10 years ago @jone No problem. Enjoy your vacation!
jone
commented
10 years ago I've taken a look at the problem and have these insights: Because the versioningbehavior was added to this test group, which adds PloneTestCase tests, the testing order is changed. This "triggers" a testing isolation problem in the caching tests.
The isolation problem is described and fixed for plone.caching in plone/plone.caching/pull/1 and fixed for plone.app.caching in plone/plone.app.caching/pull/9.
No changes in plone.app.versioningbehavior are required since this not really a problem there but only triggered because of the different test order when adding the versioningbehavior to the testing group.
tisto
commented
10 years ago Thanks @jone for figuring that out! I will look into it...
tisto
commented
10 years ago @jone Thank you so much for working on that! Your pull requests fixed quite a few issues. I owe you a beer or two at Ploneconf. :)
Both pull requests have been merged.
Though, putting p.a.versioningbehavior into the plone_app_testing group seems to cause strange and flickering test failures (e.g. ImportError: No module named CMFEditions.interfaces.IArchivist) in the other test groups (they run fully isolated in a separate "bin/test" process, so I have no idea how this happens):
http://jenkins.plone.org/job/plone-5.0-python-2.7/2805/consoleFull
Any ideas?
jone
commented
10 years ago @tisto you're welcome :wink:
I see two possible sources for the import problems:
I've checked out and run the commit locally (https://github.com/plone/buildout.coredev/commit/3ec2ea5f5b8b5f0d163f315a9a901afe14e2805 with ./bin/buildout -c jenkins.cfg && bin/jenkins-alltests --all) and it mostly passed (I've only had some robot errors because of timeouts, which are not relevant for this problem).
Did the build run multiple times? Can we run it again for checking wether it was only a temporary error? It's a little hard to debug when I'm not able to reproduce it locally :confused:
tisto
commented
10 years ago @jone I had to completely remove plone.app.linkintegrity from auto-checkout because it broke the build quite regularly. See
http://jenkins.plone.org/job/plone-5.0-python-2.7/2820/consoleFull
for the latest failure.
We use a buildout cache and I removed P.CMFEditions from all slave caches.
Let me know if there is anything I can do to assist you. I could give you access to jenkins.plone.org so you can re-add the package and trigger builds yourself if you want.
jone
commented
10 years ago @tisto I've tried to reproduce the problem locally (Mac OS) as well as on our company CI (Linux) but failed. It seems that it only occurs on the Plone Jenkins servers. Is there a VM I could clone for debugging? Otherwise I think I'd need ssh access for debugging.
tisto
commented
10 years ago We are working on a vagrant image and a ansible playbook to set up the exact configuration that jenkins.plone.org uses. I hope we can finish that by the end of the week. I will get back to you...
tomgross
commented
8 years ago Is this still an issue?
jensens
commented
6 years ago I close this one. First, the plone.app.versioningbehavior problem itself was solved. Second, I am sure any of those problems of other packages will be addressed with the Python 3 transition process.
If there are problems left and they were not solved in the python3 branches, please create fresh issues.
plone.app.versioningbehavior is currently excluded from the tests in buildout.coredev 5.0 because some tests fail and the package seems to have test isolation problems. Even though, p.a.versioningbehavior is not officially part of the core, we should test it. Therefore I would suggest to move all tests to use plone.app.testing, making sure we do not have any test isolation problems and re-include the tests into our testing setup.
cc @jone
Test failures in buildout.coredev 5.0: