search

mgedmin / dozer

WSGI middleware for profiling CPU/memory and inspecting logs
https://pypi.python.org/pypi/Dozer
Other
89 stars 12 forks source link

Non-deterministic test failure in test_count_objects #3

Open mgedmin opened 7 years ago

mgedmin commented 7 years ago

This looks like a non-deterministic test failure:

======================================================================
FAIL: test_count_objects (dozer.tests.test_reftree.TestGlobals)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/mgedmin/dozer/dozer/tests/test_reftree.py", line 61, in test_count_objects
    self.assertTrue((1, MyObj) in res)
AssertionError: False is not true

-- https://travis-ci.org/mgedmin/dozer/jobs/192265786

mgedmin commented 6 years ago

This happened again, on Python 3.5 this time:

======================================================================
FAIL: test_count_objects (dozer.tests.test_reftree.TestGlobals)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/mgedmin/dozer/dozer/tests/test_reftree.py", line 63, in test_count_objects
    self.assertIn((1, MyObj), res)
AssertionError: (1, <class 'dozer.tests.test_reftree.MyObj'>) not found in [(1, <class 'configparser.BasicInterpolation'>), (1, <class 'pkg_resources._vendor.pyparsing.White'>), (1, <class '_ast.FloorDiv'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_robotparser'>), (1, <class '_ast.GtE'>), (1, <class 'pkg_resources.EmptyProvider'>), (1, <class 'nose.plugins.manager.NoPlugins'>), (1, <class 'coverage.config.CoverageConfig'>), (1, <class '_ast.AugLoad'>), (1, <class 'six._MovedItems'>), (1, <class 'coverage.data.CoverageData'>), (1, <class 'six.Module_six_moves_urllib'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_robotparser'>), (1, <class 'pkg_resources.extern.packaging._structures.NegativeInfinity'>), (1, <class '_ast.Sub'>), (1, <class 'webob.request._NoDefault'>), (1, <class '_thread._local'>), (1, <class 'method-wrapper'>), (1, <class '_ast.AugStore'>), (1, <class 'coverage.control.Coverage'>), (1, <class '_ast.Is'>), (1, <class 'nose.plugins.failuredetail.FailureDetail'>), (1, <class 'dozer.tests.test_reftree.TestReferentTree'>), (1, <class 'multiprocessing.context.ForkContext'>), (1, <class 'mappingproxy'>), (1, <class 'nose.plugins.logcapture.LogCapture'>), (1, <class 'xml.dom.minidom.TypeInfo'>), (1, <class 'nose.plugins.capture.Capture'>), (1, <class '_ast.LtE'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib'>), (1, <class '_ast.Mult'>), (1, <class 'pkg_resources._vendor.six._SixMetaPathImporter'>), (1, <class 'dozer.tests.test_util.TestGlobals'>), (1, <class '_ast.Invert'>), (1, <class '_io.BufferedReader'>), (1, <class '_ast.NotEq'>), (1, <class '_ast.Param'>), (1, <class 'mako.util.memoized_instancemethod'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_response'>), (1, <class 'multiprocessing.context.SpawnContext'>), (1, <class '_ast.IsNot'>), (1, <class 'logging.Manager'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_parse'>), (1, <class 'nose.config.NoOptions'>), (1, <class 'pkg_resources._vendor.pyparsing._Constants'>), (1, <class 'collections.defaultdict'>), (1, <class '_ast.MatMult'>), (1, <class 'nose.plugins.cover.Coverage'>), (1, <class 'logging.RootLogger'>), (1, <class 'encodings.utf_8.IncrementalDecoder'>), (1, <class 'nose.tools.trivial.Dummy'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_response'>), (1, <class 'multiprocessing.context.ForkServerContext'>), (1, <class '_ast.Not'>), (1, <class 'six.Module_six_moves_urllib_error'>), (1, <class 'optparse.OptionParser'>), (1, <class 'nose.plugins.collect.CollectOnly'>), (1, <class 'threading._MainThread'>), (1, <class 'string._TemplateMetaclass'>), (1, <class '_ast.Div'>), (1, <class 'RecursionError'>), (1, <class 'multiprocessing.context.DefaultContext'>), (1, <class 'xml.dom.minicompat.EmptyNodeList'>), (1, <class 'doctest.DocTestFinder'>), (1, <class 'configparser.Interpolation'>), (1, <class 'unittest.runner._WritelnDecorator'>), (1, <class '_ast.UAdd'>), (1, <class '_ast.NotIn'>), (1, <class 'nose.plugins.logcapture.FilterSet'>), (1, <class 'six.Module_six_moves_urllib_request'>), (1, <class 'coverage.data.CoverageDataFiles'>), (1, <class 'bs4.builder.TreeBuilderRegistry'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib'>), (1, <class 'six._SixMetaPathImporter'>), (1, <class 'email._encoded_words._QByteMap'>), (1, <class '_ast.Mod'>), (1, <class 'nose.suite.LazySuite'>), (1, <class 'coverage.files.ModuleMatcher'>), (1, <class 'nose.loader.TestLoader'>), (1, <class 'pkg_resources._vendor.six._MovedItems'>), (1, <class 'mock.mock._Sentinel'>), (1, <class '_ast.USub'>), (1, <class 'pkg_resources._vendor.six._MovedItems'>), (1, <class 'reprlib.Repr'>), (1, <class 'nose.plugins.allmodules.AllModules'>), (1, <class 'mimetypes.MimeTypes'>), (1, <class 'pathlib._NormalAccessor'>), (1, <class 'nose.plugins.logcapture.MyMemoryHandler'>), (1, <class 'nose.plugins.prof.Profile'>), (1, <class 'json.encoder.JSONEncoder'>), (1, <class '_ast.BitOr'>), (1, <class '_ast.Pow'>), (1, <class 'xml.dom.xmlbuilder._AsyncDeprecatedProperty'>), (1, <class 'pbr.version.SemanticVersion'>), (1, <class 'nose.plugins.manager.DefaultPluginManager'>), (1, <class 'types.SimpleNamespace'>), (1, <class 'pkg_resources._vendor.pyparsing.Or'>), (1, <class 'nose.plugins.skip.Skip'>), (1, <class 'pkg_resources.ResourceManager'>), (1, <class '_ast.LShift'>), (1, <class 'dozer.reftree.CircularReferents'>), (1, <class 'nose.plugins.multiprocess.MultiProcess'>), (1, <class 'nose.plugins.testid.TestId'>), (1, <class 'six.Module_six_moves_urllib_parse'>), (1, <class 'mako.filters.Decode'>), (1, <class '_ast.Eq'>), (1, <class 'pathlib._PosixFlavour'>), (1, <class 'weakref.KeyedRef'>), (1, <class 'mock.mock._ANY'>), (1, <class 'pkg_resources._vendor.pyparsing.LineStart'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_error'>), (1, <class 'abc.abstractproperty'>), (1, <class 'nose.result.TextTestResult'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_error'>), (1, <class 'multiprocessing.process.AuthenticationString'>), (1, <class 'mako.util.PluginLoader'>), (1, <class 'mako.filters.XMLEntityEscaper'>), (1, <class 'nose.plugins.xunit.Xunit'>), (1, <class 'xml.dom.minidom.DOMImplementation'>), (1, <class '_ast.RShift'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_request'>), (1, <class 'coverage.debug.DebugOutputFile'>), (1, <class '_json.Scanner'>), (1, <class '_ast.Lt'>), (1, <class 'coverage.collector.Collector'>), (1, <class 'nose.selector.Selector'>), (1, <class 'six.Module_six_moves_urllib_response'>), (1, <class '_ast.In'>), (1, <class 'json.decoder.JSONDecoder'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_request'>), (1, <class 'coverage.debug.DebugControl'>), (1, <class '_ast.Del'>), (1, <class 'webob.etag._NoETag'>), (1, <class 'pkg_resources.extern.VendorImporter'>), (1, <class 'multiprocessing.process._MainProcess'>), (1, <class '_ast.And'>), (1, <class 'coverage.phystokens.CachedTokenizer'>), (1, <class 'pathlib._WindowsFlavour'>), (1, <class 'nose.plugins.isolate.IsolationPlugin'>), (1, <class 'nose.plugins.deprecated.Deprecated'>), (1, <class 'coverage.execfile.DummyLoader'>), (1, <class '_ast.Load'>), (1, <class 'pkg_resources._vendor.pyparsing.StringStart'>), (1, <class 'mock.mock._Call'>), (1, <class 'webob.etag._AnyETag'>), (1, <class 'coverage.cmdline.CoverageScript'>), (1, <class 'nose.plugins.attrib.AttributeSelector'>), (1, <class 'pkg_resources.WorkingSet'>), (1, <class 'functools.partial'>), (1, <class 'nose.core.TextTestRunner'>), (1, <class '_ast.Or'>), (1, <class 'webob.datetime_utils._UTC'>), (1, <class 'webtest.utils.NoDefault'>), (1, <class 'nose.plugins.debug.Pdb'>), (1, <class 'email._policybase.Compat32'>), (1, <class 'logging.StreamHandler'>), (1, <class 'nose.config.Config'>), (1, <class 'pkg_resources._vendor.pyparsing.StringEnd'>), (1, <class 'nose.suite.ContextSuiteFactory'>), (1, <class 'unittest.loader.TestLoader'>), (1, <class 'coverage.plugin_support.Plugins'>), (1, <class 'pkg_resources._vendor.pyparsing._NullToken'>), (1, <class '_ast.Store'>), (1, <class 'pkg_resources._vendor.six.Module_six_moves_urllib_parse'>), (1, <class 'nose.importer.Importer'>), (1, <class 'nose.core.TestProgram'>), (1, <class 'nose.plugins.doctests.Doctest'>), (1, <class 'pkg_resources.extern.packaging._structures.Infinity'>), (1, <class 'pkg_resources.EggInfoDistribution'>), (1, <class '_ast.BitAnd'>), (1, <class 'coverage.files.TreeMatcher'>), (1, <class 'pkg_resources._vendor.pyparsing.Dict'>), (1, <class 'site._Helper'>), (1, <class 'calendar.TextCalendar'>), (1, <class '_ast.Gt'>), (1, <class 'six.Module_six_moves_urllib_robotparser'>), (1, <class 'nose.proxy.ResultProxyFactory'>), (1, <class 'mako.runtime.Undefined'>), (1, <class 'tempfile._RandomNameSequence'>), (1, <class 'pkg_resources._vendor.six._SixMetaPathImporter'>), (1, <class 'logging._StderrHandler'>), (1, <class '_ast.Add'>), (1, <class 'pkg_resources.MemoizedZipManifests'>), (1, <class '_ast.BitXor'>), (1, <class 'codecs.CodecInfo'>), (2, <class 'ssl._ASN1Object'>), (2, <class 'pkg_resources._vendor.pyparsing.QuotedString'>), (2, <class 'webtest.app.TestRequest'>), (2, <class 'Exception'>), (2, <class 'weakref.WeakValueDictionary'>), (2, <class 'encodings.utf_8.IncrementalEncoder'>), (2, <class 'site.setquit.<locals>.Quitter'>), (2, <class 'calendar._localized_month'>), (2, <class 'dozer.logview.Logview'>), (2, <class '_io.BufferedWriter'>), (2, <class 'weakref.WeakKeyDictionary'>), (2, <class 'nose.plugins.errorclass.ErrorClass'>), (2, <enum 'PlistFormat'>), (2, <class 'threading.Thread'>), (2, <class 'calendar._localized_day'>), (2, <class 'webtest.response.TestResponse'>), (2, <enum 'Handlers'>), (2, <class 'webob.request.Request'>), (2, <class 'traceback'>), (2, <enum 'Purpose'>), (2, <class 'pkg_resources._vendor.pyparsing.Forward'>), (2, <class 'optparse.Values'>), (2, <class 'webtest.app.TestApp'>), (2, <class 'webtest.app.CookiePolicy'>), (2, <class 'webob.response.Response'>), (2, <class 'dozer.tests.test_reftree.TestGlobals'>), (2, <class 'dozer.tests.test_logview.TestRequestHandler'>), (2, <class 'mako.template.Template'>), (2, <class 'mako.template.ModuleInfo'>), (2, <class 'http.cookiejar.CookieJar'>), (2, <class 'email.charset.Charset'>), (2, <class 'xml.sax.handler.ErrorHandler'>), (2, <class '_io.BytesIO'>), (2, <class 'types.DynamicClassAttribute'>), (2, <class 'mako.lookup.TemplateLookup'>), (2, <class 'dozer.tests.test_logview.TestLogview'>), (2, <class 'dozer.leak.Dozer'>), (2, <class 'webtest.lint.InputWrapper'>), (2, <class 'contextlib._GeneratorContextManager'>), (2, <class 'webtest.lint.ErrorWrapper'>), (2, <class 'dozer.tests.test_reftree.TestTree'>), (2, <class 'os._Environ'>), (2, <class 'random.Random'>), (2, <class 'webtest.lint.IteratorWrapper'>), (2, <class 'dozer.tests.test_reftree.TestReferrerTree'>), (2, <class 'webob.dec.wsgify'>), (3, <enum 'Sigmasks'>), (3, <class 'mako.util.memoized_property'>), (3, <class 'nose.plugins.errorclass.MetaErrorClass'>), (3, <class '_io.FileIO'>), (3, <class 'logging.Formatter'>), (3, <class 'threading.Event'>), (3, <class 'logging.PlaceHolder'>), (3, <class 'collections.deque'>), (3, <class 'threading.Condition'>), (3, <class '_io.TextIOWrapper'>), (3, <class 'mock.mock._SentinelObject'>), (3, <class 'pkg_resources._vendor.pyparsing.LineEnd'>), (3, <class 'pkg_resources._vendor.pyparsing.OneOrMore'>), (3, <class 'doctest.DocTestParser'>), (3, <class 'dozer.tests.test_leak.TestReferrerTree'>), (3, <class 'logging.PercentStyle'>), (3, <class 'site._Printer'>), (3, <class 'itertools.count'>), (4, <class 'dozer.tests.test_reftree.TestCircularReferents'>), (4, <class 'webob.headers.ResponseHeaders'>), (4, <class 'pkg_resources._vendor.pyparsing.NotAny'>), (4, <class 'unittest.case._Outcome'>), (5, <class 'dozer.tests.test_reftree.MyObj'>), (5, <enum '_SSLMethod'>), (5, <class 'pkg_resources._vendor.pyparsing.Empty'>), (5, <enum '_ParameterKind'>), (5, <class '_io.StringIO'>), (5, <class 'pkg_resources._vendor.pyparsing.Group'>), (6, <class 'dozer.tests.test_dozer.TestFactories'>), (6, <class 'dozer.tests.test_logview.TestEntireStack'>), (6, <class 'functools._lru_cache_wrapper'>), (6, <class 'dozer.tests.test_profile.TestGlobals'>), (6, <class 'webob.cachecontrol.exists_property'>), (6, <class 'abc.SignalDict'>), (7, <enum 'SocketKind'>), (8, <class 'webob.cachecontrol.value_property'>), (8, <class 'string.Template'>), (9, <class 'urllib.parse.SplitResult'>), (9, <class '__future__._Feature'>), (9, <class 'coverage.cmdline.CmdOptionParser'>), (9, <class 'pkg_resources._vendor.pyparsing.ZeroOrMore'>), (9, <class 'operator.methodcaller'>), (10, <class 'dozer.logview.RequestHandler'>), (10, <class 'optparse.IndentedHelpFormatter'>), (11, <class 'dozer.tests.test_leak.TestDozer'>), (11, <class 'mako.parsetree._TagMeta'>), (12, <class 'collections.OrderedDict'>), (12, <class 'nose.proxy.ResultProxy'>), (12, <class 'logging.LogRecord'>), (12, <class 'enum.EnumMeta'>), (15, <class 'dozer.tests.test_leak.TestEntireStack'>), (18, <class 'dozer.tests.test_profile.TestEntireStack'>), (18, <class 'pkg_resources._vendor.pyparsing.Word'>), (18, <class 'pkg_resources._vendor.pyparsing.Combine'>), (19, <class 'ipaddress.IPv4Network'>), (19, <class 'PIL.ImageMode.ModeDescriptor'>), (21, <class 'operator.attrgetter'>), (22, <class 'classmethod_descriptor'>), (22, <class 'pkg_resources._vendor.pyparsing.Suppress'>), (24, <class 'pkg_resources._vendor.pyparsing.MatchFirst'>), (26, <class '_frozen_importlib_external.ExtensionFileLoader'>), (27, <class 'pkg_resources.DistInfoDistribution'>), (27, <class 'logging.Logger'>), (27, <class 'nose.suite.ContextSuite'>), (28, <class 'pkg_resources.PathMetadata'>), (28, <enum 'AddressFamily'>), (28, <class 'ipaddress.IPv6Network'>), (30, <class 'ipaddress.IPv4Address'>), (32, <enum 'Signals'>), (34, <class 'nose.plugins.manager.PluginProxy'>), (43, <class 'pkg_resources._vendor.six.MovedModule'>), (43, <class 'pkg_resources._vendor.six.MovedModule'>), (43, <class '_frozen_importlib_external.FileFinder'>), (43, <class 'ipaddress.IPv6Address'>), (43, <class 'pkg_resources._vendor.pyparsing.Optional'>), (44, <class 'six.MovedModule'>), (55, <class 'pkg_resources._vendor.pyparsing.Regex'>), (56, <class 'list_iterator'>), (57, <enum 'HTTPStatus'>), (59, <class 'frozenset'>), (62, <class 'generator'>), (66, <class 'sre_constants._NamedIntConstant'>), (81, <class 'nose.case.Test'>), (83, <class 'pkg_resources._vendor.six.MovedAttribute'>), (83, <class 'pkg_resources._vendor.six.MovedAttribute'>), (88, <class 'six.MovedAttribute'>), (89, <class 'optparse.Option'>), (91, <class 'pkg_resources.EntryPoint'>), (92, <class 'pkg_resources._vendor.pyparsing.Literal'>), (93, <class 'abc.ABCMeta'>), (103, <class 'PIL.TiffTags.TagInfo'>), (109, <class 'pkg_resources._vendor.pyparsing.And'>), (112, <class 'staticmethod'>), (131, <class 'operator.itemgetter'>), (147, <class 'frame'>), (184, <class 'classmethod'>), (213, <class 'method'>), (281, <class '_weakrefset.WeakSet'>), (375, <class '_frozen_importlib_external.SourceFileLoader'>), (431, <class '_frozen_importlib.ModuleSpec'>), (438, <class 'member_descriptor'>), (441, <class 'module'>), (631, <class 'property'>), (722, <class 'set'>), (765, <class 'cell'>), (886, <class 'builtin_function_or_method'>), (1067, <class 'method_descriptor'>), (1161, <class 'wrapper_descriptor'>), (1321, <class 'getset_descriptor'>), (1400, <class 'type'>), (2200, <class 'weakref'>), (3970, <class 'list'>), (3979, <class 'tuple'>), (4687, <class 'dict'>), (10175, <class 'function'>)]

-- https://travis-ci.org/mgedmin/dozer/jobs/365546393

Note that res contains (5, <class 'dozer.tests.test_reftree.MyObj'>), so somehow we have four more objects than we expect?

mgedmin commented 6 years ago

Actually there were more failures that I failed to notice because I don't check the Travis CI status every day and I don't have email notifications enabled:

7 failures in two years, with daily builds.