Error extracting metadata from many .csproj - KeyNotFoundException

[marcus-sonestedt]

Operation System: (Windows or Linux or MacOS)

Windows 10,

DocFX Version Used:

NuGet docfx.console 2.40.11

Template used: (default or statictoc or contain custom template)

default

Steps to Reproduce:

1. Create .NET Framework DLL project in solution with many existing .csproj project files

2. Update src in docfx.json to:

   
   "src": [
       {
         "files": [
           "**/*.csproj"
         ],
         "cwd": "..",
         "exclude": [
           "**/obj/**",
           "**/bin/**",
           "_site/**"
         ]
       }
     ],
     "dest": "obj/api"
   }

3. Build documentation project

Expected Behavior:

Build completes and gives API info

Actual Behavior:

DocFX fails with KeyNotFoundException:

1>  [19-03-08 12:47:39.367]Info:[MetadataCommand]Completed Scope:MetadataCommand in 68032.6128 milliseconds.
1>  [19-03-08 12:47:39.369]Error:Error extracting metadata for C:/Dev/cpac-eol-C#/Cpac.Core/Cpac.Core.csproj,C:/Dev/cpac-eol-C#/Cpac.Eol/Cpac.Eol.csproj,C:/Dev/cpac-eol-C#/Cpac.Eol/Cpac.Eol.Test/Cpac.Eol.Test.csproj,C:/Dev/cpac-eol-C#/Cpac.Eol.CIM/Cpac.Eol.CIM.csproj,C:/Dev/cpac-eol-C#/Cpac.Eol.Dongles/Cpac.EOL.Dongles.csproj,C:/Dev/cpac-eol-C#/Cpac.Eol.Dongles/Cpac.EOL.Dongles.NMBTGWSupport/Cpac.EOL.Dongles.NMBTGWSupport.csproj,C:/Dev/cpac-eol-C#/Cpac.Eol.SCM/Cpac.Eol.SCM.csproj,C:/Dev/cpac-eol-C#/Cpac.Eol.Smhcu/Cpac.Eol.SMHCU.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.Beckhoff/Cpac.Hw.Beckhoff.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.BLE/Cpac.Hw.Ble.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.Ixxat/Cpac.Hw.Ixxat.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.Ixxat/Cpac.Hw.Ixxat.Test/Cpac.Hw.Ixxat.Test.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.Kvaser/Cpac.Hw.Kvaser.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.NiCan/Cpac.Hw.NiCan.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.Peak/Cpac.Hw.Peak.csproj,C:/Dev/cpac-eol-C#/Cpac.Hw.Serial/Cpac.Hw.Serial.csproj,C:/Dev/cpac-eol-C#/Documentation/Documentation.csproj,C:/Dev/cpac-eol-C#/MlRuntime/MLRuntime.csproj,C:/Dev/cpac-eol-C#/Utils.BLEUtil/BLEUtil.csproj,C:/Dev/cpac-eol-C#/Utils.DST0Reader/DST0Reader.csproj,C:/Dev/cpac-eol-C#/Utils.DST0Writer/DST0Writer.csproj,C:/Dev/cpac-eol-C#/Utils.LabelSampleGenerator/LabelSampleGenerator.csproj,C:/Dev/cpac-eol-C#/Utils.SFTPUpload/SftpUpload.csproj,C:/Dev/cpac-eol-C#/Utils.VQT/VQT.csproj: 
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
1>     at System.Collections.Concurrent.ConcurrentDictionary`2.get_Item(TKey key)
1>     at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.<SaveAllMembersFromCacheAsync>d__13.MoveNext()
1>  --- End of stack trace from previous location where exception was thrown ---
1>     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
1>     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
1>     at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.<ExtractMetadataAsync>d__11.MoveNext()
1>  [19-03-08 12:47:39.369]Info:Completed in 68048.0199 milliseconds

(Full VS output here: https://pastebin.com/9UR58Dwj)

[marcus-sonestedt]

Let me know if I can help to debug this further. DocFX looks really awesome. :)

[marcus-sonestedt]

It worked when I ran "dotfx metadata Cpac.Eol.sln" first, instead of specifying all .csproj files in docfx.json.

I won't close this now as it seems like there is a bug there.

Maybe I have some .csproj-file which is not in the solution. Hm. I'll update if I find something.

[superyyrrzz]

Looks like L305 or L308 causes this error. But I cannot think of how it will trigger the error. Maybe this can help you debug the issue, or create a minimal repro docset. https://github.com/dotnet/docfx/blob/04926b3451c88be92b37571e9c7491a970690cfc/src/Microsoft.DocAsCode.Metadata.ManagedReference/ExtractMetadataWorker.cs#L305-L308

[batbaille]

I have the same issue :

C:\Solutions\FluxxDev\Fluxx\Documentation\docfx>start /b /wait .\docfx\docfx.exe metadata docfx_project/docfx.json [19-04-11 04:02:46.492]Info:[ExtractMetadata]Using msbuild C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin as inner compiler. [19-04-11 04:02:47.588]Info:[ExtractMetadata]Loading projects... [19-04-11 04:02:56.027]Info:[ExtractMetadata]Generating metadata for each project... [19-04-11 04:03:06.186]Error:Error extracting metadata for C:/Solutions/FluxxDev/Fluxx/Development/Code/Business/Fluxx.Business/FluXX.Business.csproj,C:/Solutions/FluxxDev/Fluxx/Development/Code/Business/Fluxx.Business.Archive/Fluxx.Business.Archive.csproj,C:/Solutions/FluxxDev/Fluxx/Development/Code/Business/Fluxx.Business.Site/FluXX.Business.Site.csproj,C:/Solutions/FluxxDev/Fluxx/Development/Code/Business/Fluxx.Business.Yard/Fluxx.Business.Yard.csproj,C:/Solutions/FluxxDev/Fluxx/Development/Code/Messages/Fluxx.Messages/FluXX.Messages.csproj,C:/Solutions/FluxxDev/Fluxx/Development/Code/Tools/Fluxx.Tools.Common/FluXX.Tools.Common.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Business/SAM.Business/SAM.Business.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Business/SAM.Business.Archive/SAM.Business.Archive.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Business/SAM.Business.Computed/SAM.Business.Computed.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Data/SAM.Data/SAM.Data.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Data/SAM.Data.Archive/SAM.Data.Archive.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Data/SAM.Data.NHibernate/SAM.Data.NHibernate.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Data/SAM.Data.Parameter/SAM.Data.Parameter.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Data/SAM.Data.Server/SAM.Data.Server.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Messages/SAM.Messages/SAM.Messages.csproj,C:/Solutions/FluxxDev/SAM/Development/Code/Tools/SAM.Tools.Common/SAM.Tools.Common.csproj: System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary. at System.Collections.Concurrent.ConcurrentDictionary`2.get_Item(TKey key) at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.d13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.d11.MoveNext() [19-04-11 04:03:06.189]Info:Completed in 19718,4894 milliseconds

My metadata config :

"metadata": [
    {
      "src": [
        {
          "files": [
            "Fluxx/Development/Code/**/Fluxx.Business*.csproj",
            "Fluxx/Development/Code/**/Fluxx.Messages.csproj",
            "Fluxx/Development/Code/**/Fluxx.Tools.Common.csproj",
            "SAM/Development/Code/**/SAM.Business*.csproj",
            "SAM/Development/Code/**/SAM.Data*.csproj",
            "SAM/Development/Code/**/SAM.Messages.csproj",
            "SAM/Development/Code/**/SAM.Tools.Common.csproj"
          ],
          "exclude": [
            "**/SAM.UI.ViewBase*",
            "**/Legacy/**",
            "**/bin/**",
            "**/obj/**",
            "**/UnitTest/**",
            "**/*Test*/**",
          ],
          "src": "../../../.."
        }
      ],
      "dest": "api",
      "filter": "filterConfig.yml",
      "disableGitFeatures": false,
      "disableDefaultFilter": false
    }
  ],

I have the version 2.41.

After debugging, I have a case problem. I will fix it in my project. Perhaps a new feature for docfx :)

[superyyrrzz]

@batbaille Thank you for finding out the root cause! Of course a PR is welcomed. Maybe it need to paas in OSPlatformSensitiveRelativePathComparer when constructing compilationCache.