[ROOT6, ROOT628] Assertion failure in clang::FileManager::getFileRef

[iarspider]

We are observing rare assertions failures (non-reproducible, unfortunately) in clang::FileManager::getFileRef:

cmsRun: .../interpreter/llvm/src/tools/clang/lib/Basic/FileManager.cpp:301: 
llvm::Expected<clang::FileEntryRef> clang::FileManager::getFileRef(llvm::StringRef, bool, bool): 
Assertion `Redirection.second->V.get<FileEntry *>() == &UFE && "filename from getStatValue() refers to wrong file"' failed.

Full logs: 1, 2.

It also happened once before: the only recorded instance is CMSSW_13_2_ROOT628_X_2023-06-11-2300, FWCore/Integration/TestIntegrationThinningTests unit test, the log is no longer available.

[cmsbuild]

A new Issue was created by @iarspider .

@Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar, @antoniovilela can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[makortel]

assign core

[makortel]

@pcanal @vgvassilev Could you take a look?

[cmsbuild]

New categories assigned: core

@Dr15Jones,@smuzaffar,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

[makortel]

Let's copy the full log here (since the originals will disappear at some point) From CMSSW_13_3_ROOT628_X_2023-08-16-2300 on el8_amd64_gcc11 Alignment/OfflineValidation unit test logs

cmsRun: /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/interpreter/llvm/src/tools/clang/lib/Basic/FileManager.cpp:301: llvm::Expected<clang::FileEntryRef> clang::FileManager::getFileRef(llvm::StringRef, bool, bool): Assertion `Redirection.second->V.get<FileEntry *>() == &UFE && "filename from getStatValue() refers to wrong file"' failed.

Thread 1 (Thread 0x2b8d540e17c0 (LWP 93538) "cmsRun"):
#3  0x00002b8d5559f34b in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x00002b8d53289acf in raise () from /lib64/libc.so.6
#6  0x00002b8d5325cea5 in abort () from /lib64/libc.so.6
#7  0x00002b8d5325cd79 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#8  0x00002b8d53282426 in __assert_fail () from /lib64/libc.so.6
#9  0x00002b8d5cfb8ecc in clang::FileManager::getFileRef(llvm::StringRef, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#10 0x00002b8d5cee2705 in clang::HeaderSearch::getFileAndSuggestModule(llvm::StringRef, clang::SourceLocation, clang::DirectoryEntry const*, bool, clang::Module*, clang::ModuleMap::KnownHeader*, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#11 0x00002b8d5cee2e0d in clang::DirectoryLookup::LookupFile(llvm::StringRef&, clang::HeaderSearch&, clang::SourceLocation, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool&, bool&, bool&, llvm::SmallVectorImpl<char>&, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#12 0x00002b8d5cee334e in clang::HeaderSearch::LookupFile(llvm::StringRef, clang::SourceLocation, bool, clang::DirectoryLookup const*, clang::DirectoryLookup const*&, llvm::ArrayRef<std::pair<clang::FileEntry const*, clang::DirectoryEntry const*> >, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#13 0x00002b8d5cf2109e in clang::Preprocessor::LookupFile(clang::SourceLocation, llvm::StringRef, bool, clang::DirectoryLookup const*, clang::FileEntry const*, clang::DirectoryLookup const*&, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#14 0x00002b8d5cf24670 in clang::Preprocessor::LookupHeaderIncludeOrImport(clang::DirectoryLookup const*&, llvm::StringRef&, clang::SourceLocation, clang::CharSourceRange, clang::Token const&, bool&, bool, bool&, clang::DirectoryLookup const*, clang::FileEntry const*, llvm::StringRef&, llvm::SmallVectorImpl<char>&, llvm::SmallVectorImpl<char>&, clang::ModuleMap::KnownHeader&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#15 0x00002b8d5cf273b6 in clang::Preprocessor::HandleHeaderIncludeOrImport(clang::SourceLocation, clang::Token&, clang::Token&, clang::SourceLocation, clang::DirectoryLookup const*, clang::FileEntry const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#16 0x00002b8d5cf298fd in clang::Preprocessor::HandleIncludeDirective(clang::SourceLocation, clang::Token&, clang::DirectoryLookup const*, clang::FileEntry const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#17 0x00002b8d5cf2de2d in clang::Preprocessor::HandleDirective(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#18 0x00002b8d5cef0e66 in clang::Lexer::LexTokenInternal(clang::Token&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#19 0x00002b8d5cef1a41 in clang::Lexer::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#20 0x00002b8d5cf67217 in clang::Preprocessor::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#21 0x00002b8d59ff0654 in clang::Parser::ConsumeBrace() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#22 0x00002b8d59ff8810 in clang::BalancedDelimiterTracker::consumeClose() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#23 0x00002b8d5a0234b3 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#24 0x00002b8d5a00ca9f in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributesWithRange&, clang::SourceLocation*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#25 0x00002b8d5a0c8faa in clang::Parser::ParseExternalDeclaration(clang::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#26 0x00002b8d5a0c972d in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#27 0x00002b8d597396b8 in cling::IncrementalParser::ParseInternal(llvm::StringRef) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#28 0x00002b8d5973a71d in cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#29 0x00002b8d596aebd4 in cling::Interpreter::parseForModule(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#30 0x00002b8d59490134 in ExecAutoParse (what=0x2b8d73129021 "\n#line 1 \"TrackingToolsPatternTools_xr dictionary payload\"\n\n#ifndef CMS_DICT_IMPL\n  #define CMS_DICT_IMPL 1\n#endif\n#ifndef _REENTRANT\n  #define _REENTRANT 1\n#endif\n#ifndef GNUSOURCE\n  #define GNUSOURC"..., header=false, interpreter=0x1aec700) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6326
#31 0x00002b8d594909bd in TCling::AutoParseImplRecurse (this=0x2ee6700, cls=0xf6a9160 "vector<Trajectory>", topLevel=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6431
#32 0x00002b8d59490d0c in TCling::AutoParseImplRecurse (this=0x2ee6700, cls=0x1290bef0 "edm::OneToOne<vector<Trajectory>,vector<reco::Track>,unsigned short>", topLevel=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6467
#33 0x00002b8d5949103d in TCling::AutoParse (this=0x2ee6700, cls=0x1290bef0 "edm::OneToOne<vector<Trajectory>,vector<reco::Track>,unsigned short>") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6516
#34 0x00002b8d59478787 in TClingLookupHelper__AutoParse (cname=0x1290bef0 "edm::OneToOne<vector<Trajectory>,vector<reco::Track>,unsigned short>") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:901
#35 0x00002b8d592d2cfd in ROOT::TMetaUtils::TClingLookupHelper::GetPartiallyDesugaredNameWithScopeHandling (this=0x544af40, tname=..., result=..., dropstd=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/clingutils/src/TClingUtils.cxx:581
#36 0x00002b8d51aa2a6c in TClassEdit::TSplitType::ShortType (this=0x7ffc19024470, answ=..., mode=3618) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/foundation/src/TClassEdit.cxx:454
#37 0x00002b8d51aa4105 in TClassEdit::GetNormalizedName (norm_name=..., name=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/foundation/src/TClassEdit.cxx:871
#38 0x00002b8d51ac6293 in TClass::GetClass (name=0x12de3340 "edm::AssociationMap<edm::OneToOne<std::vector<Trajectory>,std::vector<reco::Track>,unsigned short> >", load=true, silent=false, hint_pair_offset=0, hint_pair_size=0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TClass.cxx:3055
#39 0x00002b8d51ac5f1d in TClass::GetClass (name=0x12de3340 "edm::AssociationMap<edm::OneToOne<std::vector<Trajectory>,std::vector<reco::Track>,unsigned short> >", load=true, silent=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TClass.cxx:2970
#40 0x00002b8d50857799 in edm::TypeWithDict::byName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreReflection.so
#41 0x00002b8d50858015 in edm::TypeWithDict::byName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreReflection.so
#42 0x00002b8d507afdf6 in edm::BranchDescription::initFromDictionary() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libDataFormatsProvenance.so
#43 0x00002b8d507b0231 in edm::BranchDescription::BranchDescription(edm::BranchType const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::Hash<1> const&, edm::TypeWithDict const&, bool, bool, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libDataFormatsProvenance.so
#44 0x00002b8d5061995e in edm::ProductRegistryHelper::addToRegistry(__gnu_cxx::__normal_iterator<edm::ProductRegistryHelper::TypeLabelItem const*, std::vector<edm::ProductRegistryHelper::TypeLabelItem, std::allocator<edm::ProductRegistryHelper::TypeLabelItem> > > const&, __gnu_cxx::__normal_iterator<edm::ProductRegistryHelper::TypeLabelItem const*, std::vector<edm::ProductRegistryHelper::TypeLabelItem, std::allocator<edm::ProductRegistryHelper::TypeLabelItem> > > const&, edm::ModuleDescription const&, edm::ProductRegistry&, edm::ProductRegistryHelper*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#45 0x00002b8d5061a391 in edm::ProducerBase::registerProducts(edm::ProducerBase*, edm::ProductRegistry*, edm::ModuleDescription const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#46 0x00002b8dad057910 in edm::maker::ModuleHolderT<edm::stream::EDProducerAdaptorBase>::registerProductsAndCallbacks(edm::ProductRegistry*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginRecoTrackerTrackProducerPlugins.so
#47 0x00002b8d50671122 in edm::Maker::makeModule(edm::MakeModuleParams const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#48 0x00002b8d505ea629 in edm::Factory::makeModule(edm::MakeModuleParams const&, edm::ModuleTypeResolverMaker const*, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#49 0x00002b8d505f9218 in edm::ModuleRegistry::getModule(edm::MakeModuleParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#50 0x00002b8d5066f749 in edm::WorkerRegistry::getWorker(edm::WorkerParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#51 0x00002b8d5066f910 in edm::WorkerManager::getWorker(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#52 0x00002b8d5064b417 in edm::(anonymous namespace)::getWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::ParameterSet&, edm::WorkerManager&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>) [clone .lto_priv.0] () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#53 0x00002b8d506530ef in edm::StreamSchedule::fillWorkers(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::vector<edm::WorkerInPath, std::allocator<edm::WorkerInPath> >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#54 0x00002b8d50656f43 in edm::StreamSchedule::fillTrigPath(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<edm::HLTGlobalStatus>, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#55 0x00002b8d5064ecf2 in edm::StreamSchedule::StreamSchedule(std::shared_ptr<edm::TriggerResultInserter>, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::StreamID, edm::ProcessContext const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#56 0x00002b8d506367a2 in edm::Schedule::Schedule(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) () from /cvmfs/cms-ib.cern.ch/sw

---> test PVall had ERRORS

cmsRun: /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/interpreter/llvm/src/tools/clang/lib/Basic/FileManager.cpp:301: llvm::Expected<clang::FileEntryRef> clang::FileManager::getFileRef(llvm::StringRef, bool, bool): Assertion `Redirection.second->V.get<FileEntry *>() == &UFE && "filename from getStatValue() refers to wrong file"' failed.

Thread 1 (Thread 0x2b3d0de0c7c0 (LWP 93141) "cmsRun"):
#3  0x00002b3d0fc1c34b in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x00002b3d0cfb4acf in raise () from /lib64/libc.so.6
#6  0x00002b3d0cf87ea5 in abort () from /lib64/libc.so.6
#7  0x00002b3d0cf87d79 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#8  0x00002b3d0cfad426 in __assert_fail () from /lib64/libc.so.6
#9  0x00002b3d16ce5ecc in clang::FileManager::getFileRef(llvm::StringRef, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#10 0x00002b3d16c0f705 in clang::HeaderSearch::getFileAndSuggestModule(llvm::StringRef, clang::SourceLocation, clang::DirectoryEntry const*, bool, clang::Module*, clang::ModuleMap::KnownHeader*, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#11 0x00002b3d16c0fe0d in clang::DirectoryLookup::LookupFile(llvm::StringRef&, clang::HeaderSearch&, clang::SourceLocation, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool&, bool&, bool&, llvm::SmallVectorImpl<char>&, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#12 0x00002b3d16c1034e in clang::HeaderSearch::LookupFile(llvm::StringRef, clang::SourceLocation, bool, clang::DirectoryLookup const*, clang::DirectoryLookup const*&, llvm::ArrayRef<std::pair<clang::FileEntry const*, clang::DirectoryEntry const*> >, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#13 0x00002b3d16c4e09e in clang::Preprocessor::LookupFile(clang::SourceLocation, llvm::StringRef, bool, clang::DirectoryLookup const*, clang::FileEntry const*, clang::DirectoryLookup const*&, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#14 0x00002b3d16c51670 in clang::Preprocessor::LookupHeaderIncludeOrImport(clang::DirectoryLookup const*&, llvm::StringRef&, clang::SourceLocation, clang::CharSourceRange, clang::Token const&, bool&, bool, bool&, clang::DirectoryLookup const*, clang::FileEntry const*, llvm::StringRef&, llvm::SmallVectorImpl<char>&, llvm::SmallVectorImpl<char>&, clang::ModuleMap::KnownHeader&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#15 0x00002b3d16c543b6 in clang::Preprocessor::HandleHeaderIncludeOrImport(clang::SourceLocation, clang::Token&, clang::Token&, clang::SourceLocation, clang::DirectoryLookup const*, clang::FileEntry const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#16 0x00002b3d16c568fd in clang::Preprocessor::HandleIncludeDirective(clang::SourceLocation, clang::Token&, clang::DirectoryLookup const*, clang::FileEntry const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#17 0x00002b3d16c5ae2d in clang::Preprocessor::HandleDirective(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#18 0x00002b3d16c1de66 in clang::Lexer::LexTokenInternal(clang::Token&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#19 0x00002b3d16c1ea41 in clang::Lexer::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#20 0x00002b3d16c94217 in clang::Preprocessor::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#21 0x00002b3d13d1d654 in clang::Parser::ConsumeBrace() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#22 0x00002b3d13d25810 in clang::BalancedDelimiterTracker::consumeClose() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#23 0x00002b3d13d504b3 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#24 0x00002b3d13d39a9f in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributesWithRange&, clang::SourceLocation*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#25 0x00002b3d13df5faa in clang::Parser::ParseExternalDeclaration(clang::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#26 0x00002b3d13df672d in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#27 0x00002b3d134666b8 in cling::IncrementalParser::ParseInternal(llvm::StringRef) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#28 0x00002b3d1346771d in cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#29 0x00002b3d133dbbd4 in cling::Interpreter::parseForModule(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#30 0x00002b3d131bd134 in ExecAutoParse (what=0x2b3d6073d162 "\n#line 1 \"DataFormatsBTauReco_xr dictionary payload\"\n\n#ifndef CMS_DICT_IMPL\n  #define CMS_DICT_IMPL 1\n#endif\n#ifndef _REENTRANT\n  #define _REENTRANT 1\n#endif\n#ifndef GNUSOURCE\n  #define GNUSOURCE 1\n#e"..., header=false, interpreter=0x19ec700) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6326
#31 0x00002b3d131bd9bd in TCling::AutoParseImplRecurse (this=0x309e700, cls=0x91ad200 "reco::IPTagInfo<edm::RefVector<vector<reco::Track>,reco::Track,edm::refhelper::FindUsingAdvance<vector<reco::Track>,reco::Track> >,reco::JTATagInfo>", topLevel=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6431
#32 0x00002b3d131be03d in TCling::AutoParse (this=0x309e700, cls=0x91ad200 "reco::IPTagInfo<edm::RefVector<vector<reco::Track>,reco::Track,edm::refhelper::FindUsingAdvance<vector<reco::Track>,reco::Track> >,reco::JTATagInfo>") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6516
#33 0x00002b3d131a3aea in TCling__AutoParseCallback (className=0x91ad200 "reco::IPTagInfo<edm::RefVector<vector<reco::Track>,reco::Track,edm::refhelper::FindUsingAdvance<vector<reco::Track>,reco::Track> >,reco::JTATagInfo>") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:628
#34 0x00002b3d13345bdd in TClingCallbacks::LookupObject (this=0x4509620, Tag=0xe3d3650) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TClingCallbacks.cxx:604
#35 0x00002b3d133e4f50 in cling::MultiplexInterpreterCallbacks::LookupObject(clang::TagDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#36 0x00002b3d13f7ed40 in clang::MultiplexExternalSemaSource::CompleteType(clang::TagDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#37 0x00002b3d1481314b in clang::Sema::RequireCompleteTypeImpl(clang::SourceLocation, clang::QualType, clang::Sema::CompleteTypeKind, clang::Sema::TypeDiagnoser*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#38 0x00002b3d14813a45 in clang::Sema::RequireCompleteType(clang::SourceLocation, clang::QualType, clang::Sema::CompleteTypeKind, clang::Sema::TypeDiagnoser&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#39 0x00002b3d13fdfcda in clang::Sema::RequireCompleteDeclContext(clang::CXXScopeSpec&, clang::DeclContext*&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#40 0x00002b3d13fe0471 in clang::Sema::BuildCXXNestedNameSpecifier(clang::Scope*, clang::Sema::NestedNameSpecInfo&, bool, clang::CXXScopeSpec&, clang::NamedDecl*, bool, bool*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#41 0x00002b3d13fe28d4 in clang::Sema::ActOnCXXNestedNameSpecifier(clang::Scope*, clang::Sema::NestedNameSpecInfo&, bool, clang::CXXScopeSpec&, bool, bool*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#42 0x00002b3d13d766d1 in clang::Parser::ParseOptionalCXXScopeSpecifier(clang::CXXScopeSpec&, clang::OpaquePtr<clang::QualType>, bool, bool, bool*, bool, clang::IdentifierInfo**, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#43 0x00002b3d13dee7cb in clang::Parser::TryAnnotateCXXScopeToken(bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#44 0x00002b3d133efdad in cling::LookupHelper::findScope(llvm::StringRef, cling::LookupHelper::DiagSetting, clang::Type const**, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/external/el8_amd64_gcc11/lib/libCling.so
#45 0x00002b3d131c00d1 in GetClassSharedLibsForModule (cls=0xe3ad130 "reco::IPTagInfo<edm::RefVector<vector<reco::Track>,reco::Track,edm::refhelper::FindUsingAdvance<vector<reco::Track>,reco::Track> >,reco::JTATagInfo>::TrackIPData", LH=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:6957
#46 0x00002b3d131c042e in TCling::GetClassSharedLibs (this=0x309e700, cls=0xe3ad130 "reco::IPTagInfo<edm::RefVector<vector<reco::Track>,reco::Track,edm::refhelper::FindUsingAdvance<vector<reco::Track>,reco::Track> >,reco::JTATagInfo>::TrackIPData") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:7061
#47 0x00002b3d131a5e5f in TClingLookupHelper__ExistingTypeCheck (tname=..., result=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:995
#48 0x00002b3d12fff81e in ROOT::TMetaUtils::TClingLookupHelper::ExistingTypeCheck (this=0x5ec72c0, tname=..., result=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/clingutils/src/TClingUtils.cxx:509
#49 0x00002b3d0b7d1ca3 in ResolveTypedefProcessType (tname=0xe3f1410 "reco::TrackIPTagInfo::TrackIPData", cursor=33, constprefix=false, start_of_type=0, end_of_type=0, mod_start_of_type=0, modified=@0x7fff2a36805b: true, result=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/foundation/src/TClassEdit.cxx:1476
#50 0x00002b3d0b7d3718 in ResolveTypedefImpl (tname=0xe3f1410 "reco::TrackIPTagInfo::TrackIPData", len=33, cursor=@0x7fff2a36805c: 33, modified=@0x7fff2a36805b: true, result=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/foundation/src/TClassEdit.cxx:1741
#51 0x00002b3d0b7d3bf6 in TClassEdit::ResolveTypedef[abi:cxx11](char const*, bool) (tname=0xe3f1410 "reco::TrackIPTagInfo::TrackIPData") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/foundation/src/TClassEdit.cxx:1775
#52 0x00002b3d0b7cd8e3 in TClassEdit::TSplitType::ShortType (this=0x7fff2a368230, answ=..., mode=3618) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/foundation/src/TClassEdit.cxx:442
#53 0x00002b3d0b7cf105 in TClassEdit::GetNormalizedName (norm_name=..., name=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/foundation/src/TClassEdit.cxx:871
#54 0x00002b3d0b83375e in ROOT::TSchemaRule::SetSourceClass (this=0xcd21180, classname=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TSchemaRule.cxx:459
#55 0x00002b3d0b8179fc in ROOT::TGenericClassInfo::CreateRuleSet (this=0x2b3d607efa60 <ROOT::GenerateInitInstanceLocal(reco::btag::TrackIPData const*)::instance>, vect=..., ProcessReadRules=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TGenericClassInfo.cxx:324
#56 0x00002b3d0b817801 in ROOT::TGenericClassInfo::GetClass (this=0x2b3d607efa60 <ROOT::GenerateInitInstanceLocal(reco::btag::TrackIPData const*)::instance>) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TGenericClassInfo.cxx:291
#57 0x00002b3d0b7f11f5 in TClass::GetClass (name=0xd3ea060 "reco::btag::TrackIPData", load=true, silent=false, hint_pair_offset=0, hint_pair_size=0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TClass.cxx:3039
#58 0x00002b3d0b7f0f1d in TClass::GetClass (name=0xd3ea060 "reco::btag::TrackIPData", load=true, silent=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TClass.cxx:2970
#59 0x00002b3d0b113b5b in TStreamerInfo::BuildCheck (this=0xd3e6700, file=0xd156380, load=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/io/io/src/TStreamerInfo.cxx:730
#60 0x00002b3d0b0d2285 in TFile::ReadStreamerInfo (this=0xd156380) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/io/io/src/TFile.cxx:3662
#61 0x00002b3d0b0c58a1 in TFile::Init (this=0xd156380, create=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/io/io/src/TFile.cxx:865
#62 0x00002b3d105ea8f8 in TStorageFactoryFile::Initialize(char const*, char const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libIOPoolTFileAdaptor.so
#63 0x00002b3d105eaf3a in TStorageFactoryFile::TStorageFactoryFile(char const*, char const*, char const*, int, int, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libIOPoolTFileAdaptor.so
#64 0x00002b3d65d310b9 in ?? ()
#65 0x00007fff00000000 in ?? ()
#66 0x00002b3d13361e46 in std::atomic<void (*)(void*, int, void**, void*)>::operator void (*)(void*, int, void**, void*) (this=0xcc0cfe0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/gcc/11.4.1-30ebdc301ebd200f2ae0e3d880258e65/include/c++/11.4.1/atomic:433
#67 0x00002b3d1335c2f7 in TClingCallFunc::ExecWithArgsAndReturn (this=0xcc0cfc0, address=0x0, args=0x7fff2a369170, nargs=5, ret=0x7fff2a3691a8) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TClingCallFunc.cxx:1856
#68 0x00002b3d131c3142 in TCling::CallFunc_ExecWithArgsAndReturn (this=0x309e700, func=0xcc0cfc0, address=0x0, args=0x7fff2a369170, nargs=5, ret=0x7fff2a3691a8) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/metacling/src/TCling.cxx:7822
#69 0x00002b3d0b828901 in TMethodCall::Execute (this=0xa9cba00, objAddress=0x0, args=0x7fff2a369170, nargs=5, ret=0x7fff2a3691a8) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/meta/src/TMethodCall.cxx:553
#70 0x00002b3d0b0db0d6 in TPluginHandler::ExecPluginImpl<char const*, char const*, char const*, int, int> (this=0x3556700) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/base/inc/TPluginManager.h:180
#71 0x00002b3d0b0da2d2 in TPluginHandler::ExecPlugin<char const*, char const*, char const*, int, int> (this=0x3556700, nargs=5) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/core/base/inc/TPluginManager.h:218
#72 0x00002b3d0b0d4c74 in TFile::Open (url=0xce5b950 "root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/mc/RunIIWinter19PFCalibDR/Single_Pion_gun_E_2to200_13TeV_pythia8/GEN-SIM-RECO/2018ConditionsNoPU_105X_upgrade2018_realistic_v4-v1/270000/2BCBEB"..., options=0x2b3d652e9c24 "", ftitle=0x2b3d652e9c24 "", compress=101, netopt=0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-b18c19a301592425217a5c1318922f39/root-6.28.05/io/io/src/TFile.cxx:4217
#73 0x00002b3d652ab999 in edm::InputFile::InputFile(char const*, char const*, edm::InputType) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#74 0x00002b3d652dd20a in edm::RootInputFileSequence::initTheFile(bool, bool, edm::InputSource*, char const*, edm::InputType) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#75 0x00002b3d652ddf90 in edm::RootPrimaryFileSequence::RootPrimaryFileSequence(edm::ParameterSet const&, edm::PoolSource&, edm::InputFileCatalog const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#76 0x00002b3d652ae7d0 in edm::PoolSource::PoolSource(edm::ParameterSet const&, edm::InputSourceDescription const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#77 0x00002b3d652af84f in edmplugin::PluginFactory<edm::InputSource* (edm::ParameterSet const&, edm::InputSourceDescription const&)>::PMaker<edm::PoolSource>::create(edm::ParameterSet const&, edm::InputSourceDescription const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#78 0x00002b3d0a31ee73 in edm::InputSourceFactory::makeInputSource(edm::ParameterSet const&, edm::InputSourceDescription const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#79 0x00002b3d0a2b55c2 in edm::makeInput(unsigned int, edm::ParameterSet&, edm::CommonParams const&, std::shared_ptr<edm::ProductRegistry>, std::shared_ptr<edm::BranchIDListHelper>, std::shared_ptr<edm::ProcessBlockHelper> const&, std::shared_ptr<edm::ThinnedAssociationsHelper>, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::PreallocationConfiguration const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#80 0x00002b3d0a2eede3 in tbb::detail::d1::function_task<edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy)::{lambda()#2}>::execute(tbb::detail::d1::execution_data&) [clone .lto_priv.0] () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02798/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT628_X_2023-08-16-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#81 0x00002b3d0bed1a6d in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=0x2b3d0e296e00, this=0x2b3d0e28be00) at /data/cmsbld/jenkins/workspa

---> test SplitVall had ERRORS

[vgvassilev]

@iarspider we will need to go over the same ROOT + LLVM in Debug procedure here. Could you help us?

cc: @hahnjo

[iarspider]

@aandvalenzuela

[iarspider]

This also happened in unit test IOPool/Output/TestPoolOutput in CMSSW_13_3_ROOT6_X_2023-08-20-2300 IB. No backtrace was present, only the assertion failure message:

cmsRun: /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-058273dff8832a728dbe5854aca119ef/root-6.29.01/interpreter/llvm-project/clang/lib/Basic/FileManager.cpp:301: llvm::Expected<clang::FileEntryRef> clang::FileManager::getFileRef(llvm::StringRef, bool, bool): Assertion `Redirection.second->V.get<FileEntry *>() == &UFE && "filename from getStatValue() refers to wrong file"' failed.
/data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_13_3_ROOT6_X_2023-08-20-2300/src/IOPool/Output/test/TestPoolOutput.sh: line 18: 1988461 Aborted                 (core dumped) cmsRun ${LOCAL_TEST_DIR}/PoolMissingTest_cfg.py
Failure using PoolMissingTest_cfg.py: status 134

[iarspider]

Happened again in CMSSW_13_3_ROOT6_X_2023-08-28-2300

Thread 1 (Thread 0x15422f8e6c80 (LWP 1488265) "cmsRun"):
#0  0x00001542304e9f41 in poll () from /lib64/libc.so.6
#1  0x000015422df34d0f in full_read.constprop () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginFWCoreServicesPlugins.so
#2  0x000015422defc8ec in edm::service::InitRootHandlers::stacktraceFromThread() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginFWCoreServicesPlugins.so
#3  0x000015422defd34b in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x0000154230414acf in raise () from /lib64/libc.so.6
#6  0x00001542303e7ea5 in abort () from /lib64/libc.so.6
#7  0x00001542303e7d79 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#8  0x000015423040d426 in __assert_fail () from /lib64/libc.so.6
#9  0x000015422b0e060c in clang::FileManager::getFileRef(llvm::StringRef, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#10 0x000015422b009e45 in clang::HeaderSearch::getFileAndSuggestModule(llvm::StringRef, clang::SourceLocation, clang::DirectoryEntry const*, bool, clang::Module*, clang::ModuleMap::KnownHeader*, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#11 0x000015422b00a54d in clang::DirectoryLookup::LookupFile(llvm::StringRef&, clang::HeaderSearch&, clang::SourceLocation, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool&, bool&, bool&, llvm::SmallVectorImpl<char>&, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#12 0x000015422b00aa8e in clang::HeaderSearch::LookupFile(llvm::StringRef, clang::SourceLocation, bool, clang::DirectoryLookup const*, clang::DirectoryLookup const*&, llvm::ArrayRef<std::pair<clang::FileEntry const*, clang::DirectoryEntry const*> >, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#13 0x000015422b0487de in clang::Preprocessor::LookupFile(clang::SourceLocation, llvm::StringRef, bool, clang::DirectoryLookup const*, clang::FileEntry const*, clang::DirectoryLookup const*&, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#14 0x000015422b06ad9d in EvaluateHasIncludeCommon(clang::Token&, clang::IdentifierInfo*, clang::Preprocessor&, clang::DirectoryLookup const*, clang::FileEntry const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#15 0x000015422b06cb92 in clang::Preprocessor::ExpandBuiltinMacro(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#16 0x000015422b071ee8 in clang::Preprocessor::HandleMacroExpandedIdentifier(clang::Token&, clang::MacroDefinition const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#17 0x000015422b08b914 in clang::Preprocessor::HandleIdentifier(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#18 0x000015422b014ee7 in clang::Lexer::LexIdentifier(clang::Token&, char const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#19 0x000015422b016f7c in clang::Lexer::LexTokenInternal(clang::Token&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#20 0x000015422b019181 in clang::Lexer::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#21 0x000015422b08e957 in clang::Preprocessor::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#22 0x000015422b05a9eb in EvaluateValue((anonymous namespace)::PPValue&, clang::Token&, DefinedTracker&, bool, clang::Preprocessor&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#23 0x000015422b05c5d1 in clang::Preprocessor::EvaluateDirectiveExpression(clang::IdentifierInfo*&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#24 0x000015422b052e72 in clang::Preprocessor::HandleIfDirective(clang::Token&, clang::Token const&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#25 0x000015422b055542 in clang::Preprocessor::HandleDirective(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#26 0x000015422b0185a6 in clang::Lexer::LexTokenInternal(clang::Token&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#27 0x000015422b019181 in clang::Lexer::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#28 0x000015422b08e957 in clang::Preprocessor::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#29 0x0000154228118de4 in clang::Parser::ConsumeBrace() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#30 0x0000154228120fa0 in clang::BalancedDelimiterTracker::consumeClose() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#31 0x000015422814bc43 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#32 0x000015422813522f in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributesWithRange&, clang::SourceLocation*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#33 0x00001542281f173a in clang::Parser::ParseExternalDeclaration(clang::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#34 0x00001542281f1ebd in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#35 0x0000154227859328 in cling::IncrementalParser::ParseInternal(llvm::StringRef) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#36 0x000015422785a38d in cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#37 0x00001542277cac14 in cling::Interpreter::parseForModule(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT6_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#38 0x00001542275b5c46 in ExecAutoParse (what=0x154205a7f214 "\n#line 1 \"DataFormatsMuonSeed_xr dictionary payload\"\n\n#ifndef CMS_DICT_IMPL\n  #define CMS_DICT_IMPL 1\n#endif\n#ifndef _REENTRANT\n  #define _REENTRANT 1\n#endif\n#ifndef GNUSOURCE\n  #define GNUSOURCE 1\n#e"..., header=false, interpreter=0x1972380) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/metacling/src/TCling.cxx:6304
#39 0x00001542275b64cf in TCling::AutoParseImplRecurse (this=0x2a3a380, cls=0xd038b20 "vector<L2MuonTrajectorySeed>", topLevel=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/metacling/src/TCling.cxx:6409
#40 0x00001542275b681e in TCling::AutoParseImplRecurse (this=0x2a3a380, cls=0xb12b920 "edm::OneToMany<vector<L2MuonTrajectorySeed>,vector<L2MuonTrajectorySeed>,unsigned int>", topLevel=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/metacling/src/TCling.cxx:6445
#41 0x00001542275b6b4f in TCling::AutoParse (this=0x2a3a380, cls=0xb12b920 "edm::OneToMany<vector<L2MuonTrajectorySeed>,vector<L2MuonTrajectorySeed>,unsigned int>") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/metacling/src/TCling.cxx:6494
#42 0x000015422759e435 in TClingLookupHelper__AutoParse (cname=0xb12b920 "edm::OneToMany<vector<L2MuonTrajectorySeed>,vector<L2MuonTrajectorySeed>,unsigned int>") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/metacling/src/TCling.cxx:901
#43 0x00001542273f8a5d in ROOT::TMetaUtils::TClingLookupHelper::GetPartiallyDesugaredNameWithScopeHandling (this=0x4a79ac0, tname=..., result=..., dropstd=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/clingutils/src/TClingUtils.cxx:578
#44 0x00001542320cfeb8 in TClassEdit::TSplitType::ShortType (this=0x7ffff9eb6730, answ=..., mode=3618) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/foundation/src/TClassEdit.cxx:454
#45 0x00001542320d3681 in TClassEdit::ShortType[abi:cxx11](char const*, int) (typeDesc=0xd04caf0 "edm::AssociationMap<edm::OneToMany<vector<L2MuonTrajectorySeed>,vector<L2MuonTrajectorySeed>,unsigned int> >", mode=3618) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/foundation/src/TClassEdit.cxx:1312
#46 0x00001542320cfdb5 in TClassEdit::TSplitType::ShortType (this=0x7ffff9eb68d0, answ=..., mode=3618) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/foundation/src/TClassEdit.cxx:446
#47 0x00001542320d1551 in TClassEdit::GetNormalizedName (norm_name=..., name=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/foundation/src/TClassEdit.cxx:871
#48 0x00001542320f3f7b in TClass::GetClass (name=0xc290880 "edm::Wrapper<edm::AssociationMap<edm::OneToMany<vector<L2MuonTrajectorySeed>,vector<L2MuonTrajectorySeed>,unsigned int> > >", load=true, silent=false, hint_pair_offset=0, hint_pair_size=0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/meta/src/TClass.cxx:3114
#49 0x00001542320f3961 in TClass::GetClass (name=0xc290880 "edm::Wrapper<edm::AssociationMap<edm::OneToMany<vector<L2MuonTrajectorySeed>,vector<L2MuonTrajectorySeed>,unsigned int> > >", load=true, silent=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/meta/src/TClass.cxx:2970
#50 0x000015423263655d in TStreamerInfo::BuildCheck (this=0xc5f8540, file=0xc3a6a80, load=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/io/io/src/TStreamerInfo.cxx:730
#51 0x00001542325f5239 in TFile::ReadStreamerInfo (this=0xc3a6a80) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/io/io/src/TFile.cxx:3679
#52 0x00001542325e8855 in TFile::Init (this=0xc3a6a80, create=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/io/io/src/TFile.cxx:882
#53 0x000015422d4cc8f8 in TStorageFactoryFile::Initialize(char const*, char const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/libIOPoolTFileAdaptor.so
#54 0x000015422d4ccf3a in TStorageFactoryFile::TStorageFactoryFile(char const*, char const*, char const*, int, int, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/libIOPoolTFileAdaptor.so
#55 0x0000154205e320b9 in ?? ()
#56 0x00007fff00000000 in ?? ()
#57 0x00001542277567c8 in std::atomic<void (*)(void*, int, void**, void*)>::operator void (*)(void*, int, void**, void*) (this=0xbc73de0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/gcc/11.4.1-30ebdc301ebd200f2ae0e3d880258e65/include/c++/11.4.1/atomic:433
#58 0x000015422775462f in TClingCallFunc::ExecWithArgsAndReturn (this=0xbc73dc0, address=0x0, args=0x7ffff9eb7580, nargs=5, ret=0x7ffff9eb75b8) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/metacling/src/TClingCallFunc.cxx:1563
#59 0x00001542275bbcc2 in TCling::CallFunc_ExecWithArgsAndReturn (this=0x2a3a380, func=0xbc73dc0, address=0x0, args=0x7ffff9eb7580, nargs=5, ret=0x7ffff9eb75b8) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/metacling/src/TCling.cxx:7804
#60 0x000015423212b05f in TMethodCall::Execute (this=0xc130c00, objAddress=0x0, args=0x7ffff9eb7580, nargs=5, ret=0x7ffff9eb75b8) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/meta/src/TMethodCall.cxx:553
#61 0x00001542325fdf14 in TPluginHandler::ExecPluginImpl<char const*, char const*, char const*, int, int> (this=0x6253340) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/base/inc/TPluginManager.h:180
#62 0x00001542325fd1a0 in TPluginHandler::ExecPlugin<char const*, char const*, char const*, int, int> (this=0x6253340, nargs=5) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/core/base/inc/TPluginManager.h:218
#63 0x00001542325f7c28 in TFile::Open (url=0xbb59c00 "root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_13_0_11/RelValTTbar_14TeV/GEN-SIM-DIGI-RAW/131X_mcRun3_2023_realistic_forEGamma_v1_RV209-v1/2580000/0c5f48ef-554f-478b-9b58-b7bdba"..., options=0x154207b00c1c "", ftitle=0x154207b00c1c "", compress=101, netopt=0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/io/io/src/TFile.cxx:4234
#64 0x0000154207ac2b49 in edm::InputFile::InputFile(char const*, char const*, edm::InputType) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#65 0x0000154207af4466 in edm::RootInputFileSequence::initTheFile(bool, bool, edm::InputSource*, char const*, edm::InputType) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#66 0x0000154207af51e0 in edm::RootPrimaryFileSequence::RootPrimaryFileSequence(edm::ParameterSet const&, edm::PoolSource&, edm::InputFileCatalog const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#67 0x0000154207ac5970 in edm::PoolSource::PoolSource(edm::ParameterSet const&, edm::InputSourceDescription const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#68 0x0000154207ac69ef in edmplugin::PluginFactory<edm::InputSource* (edm::ParameterSet const&, edm::InputSourceDescription const&)>::PMaker<edm::PoolSource>::create(edm::ParameterSet const&, edm::InputSourceDescription const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/pluginIOPoolInput.so
#69 0x00001542333e7023 in edm::InputSourceFactory::makeInputSource(edm::ParameterSet const&, edm::InputSourceDescription const&) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#70 0x000015423337d6d2 in edm::makeInput(unsigned int, edm::ParameterSet&, edm::CommonParams const&, std::shared_ptr<edm::ProductRegistry>, std::shared_ptr<edm::BranchIDListHelper>, std::shared_ptr<edm::ProcessBlockHelper> const&, std::shared_ptr<edm::ThinnedAssociationsHelper>, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::PreallocationConfiguration const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#71 0x00001542333b6e73 in tbb::detail::d1::function_task<edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy)::{lambda()#2}>::execute(tbb::detail::d1::execution_data&) [clone .lto_priv.0] () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#72 0x000015423181da6d in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=0x15422f5f3000, this=0x15422f5e7e00) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-5caec19d753f5e33a53af823382dc5ea/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322
#73 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x15422f5e7e00) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-5caec19d753f5e33a53af823382dc5ea/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
#74 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=..., w_ctx=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-5caec19d753f5e33a53af823382dc5ea/tbb-v2021.9.0/src/tbb/task_dispatcher.cpp:168
#75 0x000015423337f066 in edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#76 0x0000154233382241 in edm::EventProcessor::EventProcessor(std::shared_ptr<edm::ProcessDesc>, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw/CMSSW_13_3_ROOT6_X_2023-08-27-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so
#77 0x000000000040c0ad in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#78 0x000015423180a877 in tbb::detail::r1::task_arena_impl::execute (ta=..., d=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-5caec19d753f5e33a53af823382dc5ea/tbb-v2021.9.0/src/tbb/arena.cpp:688
#79 0x00000000004101e0 in main::{lambda()#1}::operator()() const ()
#80 0x0000000000407d6c in main ()

[iarspider]

Also in CMSSW_13_3_ROOT628_X_2023-08-28-2300:

===========================================================
There was a crash (kSigAbort).
This is the entire stack trace of all threads:
===========================================================
#0  0x00001465ec24e5db in waitpid () from /lib64/libc.so.6
#1  0x00001465ec1b0ce7 in do_system () from /lib64/libc.so.6
#2  0x00001465eb947506 in TUnixSystem::Exec (this=0x19d86d0, shellcmd=0xe6842b0 "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/etc/gdb-backtrace.sh 2018814 1>&2") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/unix/src/TUnixSystem.cxx:2104
#3  0x00001465eb947da7 in TUnixSystem::StackTrace (this=0x19d86d0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/unix/src/TUnixSystem.cxx:2395
#4  0x00001465ebb87496 in (anonymous namespace)::do_trace (sig=5) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:182
#5  0x00001465ebb87512 in (anonymous namespace)::TExceptionHandlerImp::HandleException (this=0x42d7150, sig=5) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:195
#6  0x00001465eb94b78f in TUnixSystem::DispatchSignals (this=0x19d86d0, sig=kSigAbort) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/unix/src/TUnixSystem.cxx:3610
#7  0x00001465eb943702 in SigHandler (sig=kSigAbort) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/unix/src/TUnixSystem.cxx:395
#8  0x00001465eb94b6e5 in sighandler (sig=6) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/unix/src/TUnixSystem.cxx:3586
#9  <signal handler called>
#10 0x00001465ec1a3acf in raise () from /lib64/libc.so.6
#11 0x00001465ec176ea5 in abort () from /lib64/libc.so.6
#12 0x00001465ec176d79 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#13 0x00001465ec19c426 in __assert_fail () from /lib64/libc.so.6
#14 0x00001465e8547ecc in clang::FileManager::getFileRef(llvm::StringRef, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#15 0x00001465e8471705 in clang::HeaderSearch::getFileAndSuggestModule(llvm::StringRef, clang::SourceLocation, clang::DirectoryEntry const*, bool, clang::Module*, clang::ModuleMap::KnownHeader*, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#16 0x00001465e8471e0d in clang::DirectoryLookup::LookupFile(llvm::StringRef&, clang::HeaderSearch&, clang::SourceLocation, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool&, bool&, bool&, llvm::SmallVectorImpl<char>&, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#17 0x00001465e847234e in clang::HeaderSearch::LookupFile(llvm::StringRef, clang::SourceLocation, bool, clang::DirectoryLookup const*, clang::DirectoryLookup const*&, llvm::ArrayRef<std::pair<clang::FileEntry const*, clang::DirectoryEntry const*> >, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::Module*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#18 0x00001465e84b009e in clang::Preprocessor::LookupFile(clang::SourceLocation, llvm::StringRef, bool, clang::DirectoryLookup const*, clang::FileEntry const*, clang::DirectoryLookup const*&, llvm::SmallVectorImpl<char>*, llvm::SmallVectorImpl<char>*, clang::ModuleMap::KnownHeader*, bool*, bool*, bool, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#19 0x00001465e84d265d in EvaluateHasIncludeCommon(clang::Token&, clang::IdentifierInfo*, clang::Preprocessor&, clang::DirectoryLookup const*, clang::FileEntry const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#20 0x00001465e84d4452 in clang::Preprocessor::ExpandBuiltinMacro(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#21 0x00001465e84d97a8 in clang::Preprocessor::HandleMacroExpandedIdentifier(clang::Token&, clang::MacroDefinition const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#22 0x00001465e84f31d4 in clang::Preprocessor::HandleIdentifier(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#23 0x00001465e847c7a7 in clang::Lexer::LexIdentifier(clang::Token&, char const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#24 0x00001465e847e83c in clang::Lexer::LexTokenInternal(clang::Token&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#25 0x00001465e8480a41 in clang::Lexer::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#26 0x00001465e84f6217 in clang::Preprocessor::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#27 0x00001465e84c22ab in EvaluateValue((anonymous namespace)::PPValue&, clang::Token&, DefinedTracker&, bool, clang::Preprocessor&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#28 0x00001465e84c3e91 in clang::Preprocessor::EvaluateDirectiveExpression(clang::IdentifierInfo*&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#29 0x00001465e84ba732 in clang::Preprocessor::HandleIfDirective(clang::Token&, clang::Token const&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#30 0x00001465e84bce02 in clang::Preprocessor::HandleDirective(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#31 0x00001465e847fe66 in clang::Lexer::LexTokenInternal(clang::Token&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#32 0x00001465e8480a41 in clang::Lexer::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#33 0x00001465e84f6217 in clang::Preprocessor::Lex(clang::Token&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#34 0x00001465e557f654 in clang::Parser::ConsumeBrace() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#35 0x00001465e5587810 in clang::BalancedDelimiterTracker::consumeClose() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#36 0x00001465e55b24b3 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#37 0x00001465e559ba9f in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributesWithRange&, clang::SourceLocation*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#38 0x00001465e5657faa in clang::Parser::ParseExternalDeclaration(clang::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#39 0x00001465e565872d in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#40 0x00001465e4cc86b8 in cling::IncrementalParser::ParseInternal(llvm::StringRef) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#41 0x00001465e4cc971d in cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#42 0x00001465e4c3dbd4 in cling::Interpreter::parseForModule(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#43 0x00001465e4a1f134 in ExecAutoParse (what=0x1465c8c0f000 "\n#line 1 \"DataFormatsMuonReco_xr dictionary payload\"\n\n#ifndef CMS_DICT_IMPL\n  #define CMS_DICT_IMPL 1\n#endif\n#ifndef _REENTRANT\n  #define _REENTRANT 1\n#endif\n#ifndef GNUSOURCE\n  #define GNUSOURCE 1\n#e"..., header=false, interpreter=0x1a37980) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TCling.cxx:6326
#44 0x00001465e4a1f9bd in TCling::AutoParseImplRecurse (this=0x19e4ab0, cls=0x7ffe01fd7db0 "reco::Muon", topLevel=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TCling.cxx:6431
#45 0x00001465e4a2003d in TCling::AutoParse (this=0x19e4ab0, cls=0x7ffe01fd7db0 "reco::Muon") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TCling.cxx:6516
#46 0x00001465e4a05aea in TCling__AutoParseCallback (className=0x7ffe01fd7db0 "reco::Muon") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TCling.cxx:628
#47 0x00001465e4ba7bdd in TClingCallbacks::LookupObject (this=0x2abd7b0, Tag=0x4085a38) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TClingCallbacks.cxx:604
#48 0x00001465e4c46f50 in cling::MultiplexInterpreterCallbacks::LookupObject(clang::TagDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#49 0x00001465e57e0d40 in clang::MultiplexExternalSemaSource::CompleteType(clang::TagDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#50 0x00001465e607514b in clang::Sema::RequireCompleteTypeImpl(clang::SourceLocation, clang::QualType, clang::Sema::CompleteTypeKind, clang::Sema::TypeDiagnoser*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#51 0x00001465e6075a45 in clang::Sema::RequireCompleteType(clang::SourceLocation, clang::QualType, clang::Sema::CompleteTypeKind, clang::Sema::TypeDiagnoser&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#52 0x00001465e5841cda in clang::Sema::RequireCompleteDeclContext(clang::CXXScopeSpec&, clang::DeclContext*&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#53 0x00001465e4c4d9be in cling::getCompleteContext(clang::Decl const*, clang::ASTContext&, clang::Sema&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#54 0x00001465e4c52766 in cling::LookupHelper::findScope(llvm::StringRef, cling::LookupHelper::DiagSetting, clang::Type const**, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/external/el8_amd64_gcc11/lib/libCling.so
#55 0x00001465e4bcc520 in TClingClassInfo::TClingClassInfo (this=0x7ffe01fd8910, interp=0x1a37980, name=0x7ffe01fd8a60 "reco::Muon", intantiateTemplate=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TClingClassInfo.cxx:88
#56 0x00001465e4bcf9bd in TClingClassInfo::IsEnum (interp=0x1a37980, name=0x7ffe01fd8a60 "reco::Muon") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TClingClassInfo.cxx:820
#57 0x00001465e4a26108 in TCling::ClassInfo_IsEnum (this=0x19e4ab0, name=0x7ffe01fd8a60 "reco::Muon") at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/core/metacling/src/TCling.cxx:8239
#58 0x00001465ebb8aa87 in Cppyy::IsEnum (type_name=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:945
#59 0x00001465ebb885a3 in Cppyy::ResolveName (cppitem_name=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:406
#60 0x00001465ebb89099 in Cppyy::GetScope (sname=...) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:506
#61 0x00001465ebcd812a in CPyCppyy::CreateScopeProxy (name=..., parent=0xd039650) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/bindings/pyroot/cppyy/CPyCppyy/src/ProxyWrappers.cxx:541
#62 0x00001465ebcb0112 in CPyCppyy::meta_getattro (pyclass=0xd039650, pyname=0x1465ebf145b0) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/bindings/pyroot/cppyy/CPyCppyy/src/CPPScope.cxx:324
#63 0x00001465ed156183 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2996
#64 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x1465ebdaa420, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#65 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebe150e0, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=1, kwnames=0x0, kwargs=0x7ffe01fd9688, kwcount=0, kwstep=1, defs=0x1465ebe85c28, defcount=1, kwdefs=0x0, closure=0x0, name=0x1465ed679530, qualname=0x1465ebed9350) at Python/ceval.c:4329
#66 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#67 0x00001465ed1a57d3 in _PyObject_FastCallDictTstate (tstate=0x18bdce0, callable=0x1465ccb099d0, args=<optimized out>, nargsf=<optimized out>, kwargs=<optimized out>) at Objects/call.c:118
#68 0x00001465ed1a5ab0 in _PyObject_Call_Prepend (tstate=tstate
entry=0x18bdce0, callable=callable
entry=0x1465ccb099d0, obj=obj
entry=0x1465ebf3f370, args=args
entry=0x1465ed67d040, kwargs=kwargs
entry=0x0) at Objects/call.c:489
#69 0x00001465ed20aa5a in slot_tp_init (self=0x1465ebf3f370, args=0x1465ed67d040, kwds=0x0) at Objects/typeobject.c:6971
#70 0x00001465ed202e8a in type_call (type=<optimized out>, args=0x1465ed67d040, kwds=0x0) at Objects/typeobject.c:1028
#71 0x00001465ed1a5668 in _PyObject_MakeTpCall (tstate=0x18bdce0, callable=0x5540600, args=<optimized out>, nargs=0, keywords=0x0) at Objects/call.c:191
#72 0x00001465ed159bfc in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775808, args=0x1465ebed6a40, callable=0x5540600, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#73 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775808, args=0x1465ebed6a40, callable=0x5540600, tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#74 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775808, args=0x1465ebed6a40, callable=0x5540600) at ./Include/cpython/abstract.h:127
#75 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#76 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3520
#77 0x00001465ed152bab in _PyEval_EvalFrame (throwflag=0, f=0x1465ebed68b0, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#78 function_code_fastcall (tstate=0x18bdce0, co=<optimized out>, args=<optimized out>, nargs=1, globals=<optimized out>) at Objects/call.c:330
#79 0x00001465ed1a81e8 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x1465ebe90b78, callable=0x1465ebf1caf0, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#80 method_vectorcall (method=<optimized out>, args=0x1465ebe90b80, nargsf=<optimized out>, kwnames=0x0) at Objects/classobject.c:53
#81 0x00001465ed159213 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x1465ebe90b80, callable=0x1465ebf42a80, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#82 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x1465ebe90b80, callable=0x1465ebf42a80) at ./Include/cpython/abstract.h:127
#83 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#84 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3520
#85 0x00001465ed152bab in _PyEval_EvalFrame (throwflag=0, f=0x1465ebe90a00, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#86 function_code_fastcall (tstate=0x18bdce0, co=<optimized out>, args=<optimized out>, nargs=2, globals=<optimized out>) at Objects/call.c:330
#87 0x00001465ed15a770 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x1917da8, callable=0x1465ebf824c0, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#88 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x1917da8, callable=0x1465ebf824c0) at ./Include/cpython/abstract.h:127
#89 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#90 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3506
#91 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x1917be0, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#92 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebfaf450, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=2, kwnames=0x0, kwargs=0x7ffe01fd9d90, kwcount=0, kwstep=1, defs=0x1465ec025208, defcount=1, kwdefs=0x0, closure=0x0, name=0x1465ed63eb30, qualname=0x1465ebfb1470) at Python/ceval.c:4329
#93 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#94 0x00001465ed1a8164 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7ffe01fd9d80, callable=0x1465ebf82670, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#95 method_vectorcall (method=<optimized out>, args=0x1465ebe78ef8, nargsf=<optimized out>, kwnames=0x0) at Objects/classobject.c:83
#96 0x00001465ed158214 in do_call_core (kwdict=0x1465ebf42c00, callargs=0x1465ebe78ee0, func=0x1465ebf42b00, tstate=0x18bdce0) at Python/ceval.c:5125
#97 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3582
#98 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x1465ebe88e40, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#99 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebfaf660, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=2, kwnames=0x0, kwargs=0x7ffe01fda0d0, kwcount=0, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x1465ed6791f0, qualname=0x1465ebfb2c10) at Python/ceval.c:4329
#100 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#101 0x00001465ed1a57d3 in _PyObject_FastCallDictTstate (tstate=0x18bdce0, callable=0x1465ebf82820, args=<optimized out>, nargsf=<optimized out>, kwargs=<optimized out>) at Objects/call.c:118
#102 0x00001465ed1a5ab0 in _PyObject_Call_Prepend (tstate=tstate
entry=0x18bdce0, callable=callable
entry=0x1465ebf82820, obj=obj
entry=0x1465ebf3f1f0, args=args
entry=0x1465ebe78dc0, kwargs=kwargs
entry=0x0) at Objects/call.c:489
#103 0x00001465ed207941 in slot_tp_call (self=0x1465ebf3f1f0, args=0x1465ebe78dc0, kwds=0x0) at Objects/typeobject.c:6731
#104 0x00001465ed1a5668 in _PyObject_MakeTpCall (tstate=0x18bdce0, callable=0x1465ebf3f1f0, args=<optimized out>, nargs=1, keywords=0x0) at Objects/call.c:191
#105 0x00001465ed159bfc in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x1465ebf0bf08, callable=0x1465ebf3f1f0, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#106 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x1465ebf0bf08, callable=0x1465ebf3f1f0, tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#107 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=0x1465ebf0bf08, callable=0x1465ebf3f1f0) at ./Include/cpython/abstract.h:127
#108 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#109 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3520
#110 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x1465ebf0bd60, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#111 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebf7fea0, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=2, kwnames=0x0, kwargs=0x7ffe01fda450, kwcount=0, kwstep=1, defs=0x1465ed5568f8, defcount=1, kwdefs=0x0, closure=0x0, name=0x1465ed63eb30, qualname=0x1465ebfbdb30) at Python/ceval.c:4329
#112 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#113 0x00001465ed1a8164 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7ffe01fda440, callable=0x1465ebf85670, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#114 method_vectorcall (method=<optimized out>, args=0x1465ebeff2c8, nargsf=<optimized out>, kwnames=0x0) at Objects/classobject.c:83
#115 0x00001465ed158214 in do_call_core (kwdict=0x1465ebf5dc40, callargs=0x1465ebeff2b0, func=0x1465ebf42e00, tstate=0x18bdce0) at Python/ceval.c:5125
#116 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3582
#117 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x1465ebe88740, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#118 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebf983a0, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=2, kwnames=0x0, kwargs=0x7ffe01fda790, kwcount=0, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x1465ed6791f0, qualname=0x1465ebf81620) at Python/ceval.c:4329
#119 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#120 0x00001465ed1a57d3 in _PyObject_FastCallDictTstate (tstate=0x18bdce0, callable=0x1465ebf85550, args=<optimized out>, nargsf=<optimized out>, kwargs=<optimized out>) at Objects/call.c:118
#121 0x00001465ed1a5ab0 in _PyObject_Call_Prepend (tstate=tstate
entry=0x18bdce0, callable=callable
entry=0x1465ebf85550, obj=obj
entry=0x1465ebf3f160, args=args
entry=0x1465ed56eaf0, kwargs=kwargs
entry=0x0) at Objects/call.c:489
#122 0x00001465ed207941 in slot_tp_call (self=0x1465ebf3f160, args=0x1465ed56eaf0, kwds=0x0) at Objects/typeobject.c:6731
#123 0x00001465ed1a5668 in _PyObject_MakeTpCall (tstate=0x18bdce0, callable=0x1465ebf3f160, args=<optimized out>, nargs=1, keywords=0x0) at Objects/call.c:191
#124 0x00001465ed159bfc in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x1465ebf0bd28, callable=0x1465ebf3f160, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#125 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x1465ebf0bd28, callable=0x1465ebf3f160, tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#126 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=0x1465ebf0bd28, callable=0x1465ebf3f160) at ./Include/cpython/abstract.h:127
#127 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#128 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3520
#129 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x1465ebf0bb80, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#130 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebf7fea0, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=2, kwnames=0x0, kwargs=0x7ffe01fdab10, kwcount=0, kwstep=1, defs=0x1465ed5568f8, defcount=1, kwdefs=0x0, closure=0x0, name=0x1465ed63eb30, qualname=0x1465ebfbdb30) at Python/ceval.c:4329
#131 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#132 0x00001465ed1a8164 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7ffe01fdab00, callable=0x1465ebf85670, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#133 method_vectorcall (method=<optimized out>, args=0x1465ed52e238, nargsf=<optimized out>, kwnames=0x0) at Objects/classobject.c:83
#134 0x00001465ed158214 in do_call_core (kwdict=0x1465ebf42dc0, callargs=0x1465ed52e220, func=0x1465ebf42c40, tstate=0x18bdce0) at Python/ceval.c:5125
#135 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3582
#136 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x1465ebe88580, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#137 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebf983a0, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=2, kwnames=0x0, kwargs=0x7ffe01fdae50, kwcount=0, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x1465ed6791f0, qualname=0x1465ebf81620) at Python/ceval.c:4329
#138 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#139 0x00001465ed1a57d3 in _PyObject_FastCallDictTstate (tstate=0x18bdce0, callable=0x1465ebf85550, args=<optimized out>, nargsf=<optimized out>, kwargs=<optimized out>) at Objects/call.c:118
#140 0x00001465ed1a5ab0 in _PyObject_Call_Prepend (tstate=tstate
entry=0x18bdce0, callable=callable
entry=0x1465ebf85550, obj=obj
entry=0x1465ed52ef70, args=args
entry=0x1465ed522220, kwargs=kwargs
entry=0x0) at Objects/call.c:489
#141 0x00001465ed207941 in slot_tp_call (self=0x1465ed52ef70, args=0x1465ed522220, kwds=0x0) at Objects/typeobject.c:6731
#142 0x00001465ed1a5668 in _PyObject_MakeTpCall (tstate=0x18bdce0, callable=0x1465ed52ef70, args=<optimized out>, nargs=1, keywords=0x0) at Objects/call.c:191
#143 0x00001465ed159bfc in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x19e7d38, callable=0x1465ed52ef70, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#144 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775809, args=0x19e7d38, callable=0x1465ed52ef70, tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#145 PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775809, args=0x19e7d38, callable=0x1465ed52ef70) at ./Include/cpython/abstract.h:127
#146 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#147 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3520
#148 0x00001465ed152bab in _PyEval_EvalFrame (throwflag=0, f=0x19e7b40, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#149 function_code_fastcall (tstate=0x18bdce0, co=<optimized out>, args=<optimized out>, nargs=2, globals=<optimized out>) at Objects/call.c:330
#150 0x00001465ed15a770 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x1465ebf0b760, callable=0x1465ebf1c310, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#151 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x1465ebf0b760, callable=0x1465ebf1c310) at ./Include/cpython/abstract.h:127
#152 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#153 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3506
#154 0x00001465ed152bab in _PyEval_EvalFrame (throwflag=0, f=0x1465ebf0b5e0, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#155 function_code_fastcall (tstate=0x18bdce0, co=<optimized out>, args=<optimized out>, nargs=1, globals=<optimized out>) at Objects/call.c:330
#156 0x00001465ed15a770 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=<optimized out>, args=0x19bdd00, callable=0x1465ebf1c9d0, tstate=0x18bdce0) at ./Include/cpython/abstract.h:118
#157 PyObject_Vectorcall (kwnames=0x0, nargsf=<optimized out>, args=0x19bdd00, callable=0x1465ebf1c9d0) at ./Include/cpython/abstract.h:127
#158 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#159 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3506
#160 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x19bdb20, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#161 _PyEval_EvalCode (tstate=tstate
entry=0x18bdce0, _co=_co
entry=0x1465ebf3c870, globals=<optimized out>, locals=locals
entry=0x0, args=<optimized out>, argcount=1, kwnames=0x1465ed511f88, kwargs=0x1465ec024520, kwcount=1, kwstep=1, defs=0x1465ebf3e5d8, defcount=11, kwdefs=0x1465ebf5dc80, closure=0x0, name=0x1465ed679530, qualname=0x1465ebf3d800) at Python/ceval.c:4329
#162 0x00001465ed1a53f1 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:396
#163 0x00001465ed1a584e in _PyObject_FastCallDictTstate (tstate=0x18bdce0, callable=0x1465ebf1c430, args=<optimized out>, nargsf=<optimized out>, kwargs=<optimized out>) at Objects/call.c:129
#164 0x00001465ed1a5ab0 in _PyObject_Call_Prepend (tstate=tstate
entry=0x18bdce0, callable=callable
entry=0x1465ebf1c430, obj=obj
entry=0x1465ed588a60, args=args
entry=0x1465ed67d040, kwargs=kwargs
entry=0x1465ed598180) at Objects/call.c:489
#165 0x00001465ed20aa5a in slot_tp_init (self=0x1465ed588a60, args=0x1465ed67d040, kwds=0x1465ed598180) at Objects/typeobject.c:6971
#166 0x00001465ed202e8a in type_call (type=<optimized out>, args=0x1465ed67d040, kwds=0x1465ed598180) at Objects/typeobject.c:1028
#167 0x00001465ed1a5668 in _PyObject_MakeTpCall (tstate=0x18bdce0, callable=0x1976a90, args=<optimized out>, nargs=0, keywords=0x1465ed511ee0) at Objects/call.c:191
#168 0x00001465ed15a0c9 in _PyObject_VectorcallTstate (kwnames=0x1465ed511ee0, nargsf=<optimized out>, args=0x19c5100, callable=0x1976a90, tstate=<optimized out>) at ./Include/cpython/abstract.h:116
#169 _PyObject_VectorcallTstate (kwnames=0x1465ed511ee0, nargsf=<optimized out>, args=0x19c5100, callable=0x1976a90, tstate=<optimized out>) at ./Include/cpython/abstract.h:103
#170 PyObject_Vectorcall (kwnames=0x1465ed511ee0, nargsf=<optimized out>, args=0x19c5100, callable=0x1976a90) at ./Include/cpython/abstract.h:127
#171 call_function (kwnames=0x1465ed511ee0, oparg=<optimized out>, pp_stack=<synthetic pointer>, tstate=0x18bdce0) at Python/ceval.c:5077
#172 _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3537
#173 0x00001465ed288c2e in _PyEval_EvalFrame (throwflag=0, f=0x19c4f90, tstate=0x18bdce0) at ./Include/internal/pycore_ceval.h:40
#174 _PyEval_EvalCode (tstate=0x18bdce0, _co=_co
entry=0x1465ed54b5b0, globals=globals
entry=0x1465ed59e5c0, locals=locals
entry=0x1465ed59e5c0, args=args
entry=0x0, argcount=argcount
entry=0, kwnames=0x0, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4329
#175 0x00001465ed288f6e in _PyEval_EvalCodeWithName (_co=_co
entry=0x1465ed54b5b0, globals=globals
entry=0x1465ed59e5c0, locals=locals
entry=0x1465ed59e5c0, args=args
entry=0x0, argcount=argcount
entry=0, kwnames=kwnames
entry=0x0, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at Python/ceval.c:4361
#176 0x00001465ed288fbb in PyEval_EvalCodeEx (_co=_co
entry=0x1465ed54b5b0, globals=globals
entry=0x1465ed59e5c0, locals=locals
entry=0x1465ed59e5c0, args=args
entry=0x0, argcount=argcount
entry=0, kws=kws
entry=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:4377
#177 0x00001465ed288feb in PyEval_EvalCode (co=co
entry=0x1465ed54b5b0, globals=globals
entry=0x1465ed59e5c0, locals=locals
entry=0x1465ed59e5c0) at Python/ceval.c:828
#178 0x00001465ed2c938e in run_eval_code_obj (locals=0x1465ed59e5c0, globals=0x1465ed59e5c0, co=0x1465ed54b5b0, tstate=0x18bdce0) at Python/pythonrun.c:1221
#179 run_mod (mod=<optimized out>, filename=filename
entry=0x1465ed50a7b0, globals=globals
entry=0x1465ed59e5c0, locals=locals
entry=0x1465ed59e5c0, flags=flags
entry=0x7ffe01fdb998, arena=arena
entry=0x1465ec096fd0) at Python/pythonrun.c:1242
#180 0x00001465ed2cada3 in pyrun_file (flags=0x7ffe01fdb998, closeit=<optimized out>, locals=0x1465ed59e5c0, globals=0x1465ed59e5c0, start=257, filename=0x1465ed50a7b0, fp=0x18b79a0) at Python/pythonrun.c:1140
#181 pyrun_simple_file (flags=0x7ffe01fdb998, closeit=<optimized out>, filename=0x1465ed50a7b0, fp=0x18b79a0) at Python/pythonrun.c:450
#182 PyRun_SimpleFileExFlags (fp=fp
entry=0x18b79a0, filename=<optimized out>, closeit=closeit
entry=1, flags=flags
entry=0x7ffe01fdb998) at Python/pythonrun.c:483
#183 0x00001465ed2cb3dc in PyRun_AnyFileExFlags (fp=fp
entry=0x18b79a0, filename=<optimized out>, closeit=closeit
entry=1, flags=flags
entry=0x7ffe01fdb998) at Python/pythonrun.c:92
#184 0x00001465ed2e8528 in pymain_run_file (cf=0x7ffe01fdb998, config=0x18bf110) at Modules/main.c:373
#185 pymain_run_python (exitcode=0x7ffe01fdb990) at Modules/main.c:598
#186 Py_RunMain () at Modules/main.c:677
#187 0x00001465ed2e89e7 in pymain_main (args=0x7ffe01fdba80) at Modules/main.c:707
#188 Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:731
#189 0x00001465ec18fd85 in __libc_start_main () from /lib64/libc.so.6
#190 0x000000000040107e in _start ()
===========================================================

[dan131riley]

The first of the previous two was in testTriggerEventAnalyzers, with the stack trace preceded by:

29-Aug-2023 04:40:59 CEST  Initiating request to open file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_13_0_11/RelValTTbar_14TeV/GEN-SIM-DIGI-RAW/131X_mcRun3_2023_realistic_forEGamma_v1_RV209-v1/2580000/0c5f48ef-554f-478b-9b58-b7bdba0d65bb.root
cmsRun: /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.29.01-bdb7734a9d2c42b06b77e8bcbf70d450/root-6.29.01/interpreter/llvm-project/clang/lib/Basic/FileManager.cpp:301: llvm::Expected<clang::FileEntryRef> clang::FileManager::getFileRef(llvm::StringRef, bool, bool): Assertion `Redirection.second->V.get<FileEntry *>() == &UFE && "filename from getStatValue() refers to wrong file"' failed.

The second was from testPhysicsToolsSelectorUtilsPythonTestsDriver, and was preceded by:

test_vid_selectors (__main__.TestVIDSelectros) ... /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/python/DataFormats/FWLite/__init__.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02800/el8_amd64_gcc11/cms/cmssw-patch/CMSSW_13_3_ROOT628_X_2023-08-28-2300/src/DataFormats/FWLite/python/__init__.py' mode='r' encoding='UTF-8'>
  exec(open(pyinit).read())
ResourceWarning: Enable tracemalloc to get the object allocation traceback
python3: /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/el8_amd64_gcc11/lcg/root/6.28.05-fddc0df05f49bc5db67a2c9871b52717/root-6.28.05/interpreter/llvm/src/tools/clang/lib/Basic/FileManager.cpp:301: llvm::Expected<clang::FileEntryRef> clang::FileManager::getFileRef(llvm::StringRef, bool, bool): Assertion `Redirection.second->V.get<FileEntry *>() == &UFE && "filename from getStatValue() refers to wrong file"' failed.