dotnet-issue-labeler[bot]
commented
3 years ago I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.
Open Anipik opened 3 years ago
dotnet-issue-labeler[bot]
commented
3 years ago I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.
hoyosjs
commented
3 years ago Looks like this can get caused by locks on the file caused by parallel compilation, antiviruses, and parallel builds: https://developercommunity.visualstudio.com/content/problem/552999/fatal-error-c1090-pdb-api-call-failed-error-code-3.html
There was a workaround in a Jenkins forum should we see this again: https://issues.jenkins.io/browse/JENKINS-9104
ericstj
commented
3 years ago Be careful about assuming we should workaround such issues. If they appear regularly it's more likely we have a concurrency issue in the build, or a bad practice (eg: open/re-write in place) in a build process.
ericstj
commented
3 years ago Just hit this again: https://dev.azure.com/dnceng/public/_build/results?buildId=1070066&view=logs&j=e93c566c-b6be-5acb-6fc2-de2972d01fec&t=2dc174be-b57a-56b6-9566-e621f39fb1f6&l=3009
block.cpp
D:\workspace\_work\1\s\src\coreclr\jit\emit.cpp(8624,1): fatal error C1090: PDB API call failed, error code '23': (0x000006BA) [D:\workspace\_work\1\s\artifacts\obj\coreclr\windows.x86.Debug\jit\clrjit_win_x86_x86.vcxproj]
hashbv.cppIt's at a different point in the build.
hoyosjs
commented
3 years ago Looks like they got more hits: https://developercommunity.visualstudio.com/t/C1090-PDB-API-call-failed-error-code-2/48897#T-ND770588 and they are starting to see this data becoming hotter in Watson. I don't know if we have the buildpool configured for heapdumps (@dnceng), but maybe setting failfast on our builds could help us get dumps if they could be useful for the MSVC team.
jeffhandley
commented
3 years ago I just hit this on #52236. https://dev.azure.com/dnceng/public/_build/results?buildId=1122253&view=logs&jobId=16bd8e89-36f9-5867-227e-26683d5e0a13&j=16bd8e89-36f9-5867-227e-26683d5e0a13&t=d2c0465b-73ac-55f7-de66-3542be19c102
2021-05-05T00:23:07.5681271Z Finished generating code
2021-05-05T00:23:07.6306732Z tocfile.cpp
2021-05-05T00:23:07.6930869Z D:\workspace\_work\1\s\src\coreclr\ToolBox\superpmi\superpmi-shared\spmiutil.cpp(341,1): fatal error C1090: PDB API call failed, error code '23': (0x00000006) [D:\workspace\_work\1\s\artifacts\obj\coreclr\windows.x86.Release\ide\ToolBox\superpmi\superpmi-shim-collector\superpmi-shim-collector.vcxproj]
2021-05-05T00:23:07.7556746Z D:\workspace\_work\1\s\src\coreclr\ToolBox\superpmi\superpmi-shared\methodcontextreader.cpp(581,1): fatal error C1090: PDB API call failed, error code '23': (0x00000006) [D:\workspace\_work\1\s\artifacts\obj\coreclr\windows.x86.Release\ide\ToolBox\superpmi\superpmi-shim-collector\superpmi-shim-collector.vcxproj]
2021-05-05T00:23:07.8189417Z typeutils.cpp
2021-05-05T00:23:07.8806507Z spmidumphelper.cpp
2021-05-05T00:23:07.9745654Z D:\workspace\_work\1\s\src\coreclr\ToolBox\superpmi\superpmi-shared\methodcontext.cpp(7113,1): fatal error C1090: PDB API call failed, error code '23': (0x00000006) [D:\workspace\_work\1\s\artifacts\obj\coreclr\windows.x86.Release\ide\ToolBox\superpmi\superpmi-shim-collector\superpmi-shim-collector.vcxproj]
2021-05-05T00:23:08.0215020Z ildasm.vcxproj -> D:\workspace\_work\1\s\artifacts\obj\coreclr\windows.x86.Release\ide\ildasm\exe\Release\ildasm.exe
2021-05-05T00:23:08.0839181Z Creating library D:/workspace/_work/1/s/artifacts/obj/coreclr/windows.x86.Release/ide/jit/Release/clrjit_unix_arm_x86.lib and object D:/workspace/_work/1/s/artifacts/obj/coreclr/windows.x86.Release/ide/jit/Release/clrjit_unix_arm_x86.exp
2021-05-05T00:23:08.1777655Z Generating code
2021-05-05T00:23:08.2727442Z Creating library D:/workspace/_work/1/s/artifacts/obj/coreclr/windows.x86.Release/ide/dlls/dbgshim/Release/dbgshim.lib and object D:/workspace/_work/1/s/artifacts/obj/coreclr/windows.x86.Release/ide/dlls/dbgshim/Release/dbgshim.exp
2021-05-05T00:23:08.3653532Z D:\workspace\_work\1\s\src\coreclr\ToolBox\superpmi\superpmi-shared\methodcontext.cpp(7113,1): fatal error C1090: PDB API call failed, error code '23': (0x000006BA) [D:\workspace\_work\1\s\artifacts\obj\coreclr\windows.x86.Release\ide\ToolBox\superpmi\superpmi\superpmi.vcxproj]Thanks @jeffhandley. I've gotten two dumps (one from your PR and one from Maoni's PR). After looking, this doesn't seem to be an issue on our infrastructure and I've routed it to the C++ team for help.
jkoritzinsky
commented
3 years ago We've started hitting this on official builds: https://dev.azure.com/dnceng/internal/_build/results?buildId=1223231&view=results
lewing
commented
3 years ago 
elinor-fung
commented
2 years ago 
danmoseley
commented
2 years ago @hoyosjs did you hear back from C++?
hoyosjs
commented
2 years ago Yeah, I talked to YongKang and we could only tell that it was corruption on an object's vtable. The suggestion was to use TTT to pinpoint where the corruption happened, but this isn't particularly easy to achieve on the build machines (slows things, files are large, and our builds are already quite bulky).
mangod9
commented
2 years ago Does a retry usually make things work?
hoyosjs
commented
2 years ago Yes
build https://dev.azure.com/dnceng/internal/_build/results?buildId=984686&view=logs&j=128f4634-3e62-52f1-6764-cb4c2b8330d4&t=32fccdd3-9e64-5af0-1f0d-f65712c6c51a
cc @dotnet/runtime-infrastructure
Report
Summary
Known issue validation
Build: :mag_right: Result validation: :warning: Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: :mag_right:" line.