search

dotnet / docfx

Static site generator for .NET API documentation.
https://dotnet.github.io/docfx/
MIT License
4.04k stars 860 forks source link

ArgumentOutOfRangeException on metadata task #4341

Open brennon opened 5 years ago

brennon commented 5 years ago

Operation System: (Windows or Linux or MacOS) Windows

DocFX Version Used: 2.41.0.0

Template used: (default or statictoc or contain custom template) default

Steps to Reproduce:

  1. Run docfx with this config against my project with a whole lot of .csproj files: 
    {
"metadata": [
{
  "src": [
    {
      "files": [
        "**/*.csproj"
      ],
      "exclude": [
        "**/*Test*/*"
      ],
      "src": "../../"
    }
  ],
  "dest": "api",
  "disableGitFeatures": true,
  "disableDefaultFilter": false
}
],
"build": {
"content": [
  {
    "files": [
      "api/**.yml",
      "api/index.md"
    ]
  },
  {
    "files": [
      "toc.yml",
      "*.md"
    ]
  }
],
"resource": [
  {
    "files": [
      "images/**"
    ]
  }
],
"overwrite": [
  {
    "files": [
      "apidoc/**.md"
    ],
    "exclude": [
      "obj/**",
      "_site/**"
    ]
  }
],
"dest": "site",
"globalMetadataFiles": [],
"fileMetadataFiles": [],
"template": [
  "default"
],
"postProcessors": [],
"markdownEngineName": "markdig",
"noLangKeyword": false,
"keepFileLink": false,
"cleanupCacheHistory": false,
"disableGitFeatures": false
}
}

Expected Behavior: Metadata is generated.

Actual Behavior: Get the following exception:

[19-04-09 08:15:33.021]Error:Error extracting metadata for D:/TeamCity/BuildAgent/work/c0a0e3e2b15ea307/ProductName/UI/UI.csproj: System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
   at System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource)
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SpecIdHelper.<>c__DisplayClass4_0.<SpecTypeGenericParameter>b__0(Match match)
   at System.Text.RegularExpressions.RegexReplacement.Replace(MatchEvaluator evaluator, Regex regex, String input, Int32 count, Int32 startat)
   at System.Text.RegularExpressions.Regex.Replace(String input, MatchEvaluator evaluator)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SpecIdHelper.GetSpecId(ISymbol symbol, IReadOnlyList`1 typeGenericParameters, IReadOnlyList`1 methodGenericParameters)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.AddInheritedMembers(INamedTypeSymbol symbol, INamedTypeSymbol type, Dictionary`2 dict, IReadOnlyList`1 typeParamterNames)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.GenerateInheritance(INamedTypeSymbol symbol, MetadataItem item)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitNamedType(INamedTypeSymbol symbol)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitDescendants[T](IEnumerable`1 children, Func`2 getChildren, Func`2 filter)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitNamespace(INamespaceSymbol symbol)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitDescendants[T](IEnumerable`1 children, Func`2 getChildren, Func`2 filter)
   at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitAssembly(IAssemblySymbol symbol)
   at Microsoft.DocAsCode.Metadata.ManagedReference.RoslynMetadataExtractor.Extract(ExtractMetadataOptions options)
   at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.GetMetadataFromProjectLevelCache(IBuildController controller, IInputParameters key)
   at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.<SaveAllMembersFromCacheAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.<ExtractMetadataAsync>d__11.MoveNext()
brennon commented 5 years ago

I'm sorry that's the best I can report. With an uninformative exception, it's hard to know how to give more information without going in and pulling hundreds of source files out of the project one by one.

KalleOlaviNiemitalo commented 5 years ago

The stack trace suggests that the exception is thrown in names[int.Parse(match.Value.Substring(1))] within SpecIdHelper.SpecTypeGenericParameter. It means the id contains "`1" syntax but the number is out of range for typeGenericParameters.

This is getting called from AddInheritedMembers, so I think the bug is triggered by a method. The generic parameters are for a type, however.

I tried a number of scenarios with inheritance and nested generic types but could not make DocFX crash. I think the bug will be easiest to track down if you run DocFX on your projects in a debugger and check the local variables in the stack frames when the exception is thrown. Even if you cannot build DocFX from source and cannot get debug symbols, this should still be doable using WinDbg and SOS.

ReubenBond commented 4 years ago

I hit this today while trying to add generated reference docs to Orleans.

The exception is in SpecTypeGenericParameter. The Debug.Assert above that line does trigger while debugging docfx metadata.

Maybe the locals will help:

match.Value = "`0"
id = "Orleans.Grain.RegisterTimer(Func{`0,`1},System.Object,TimeSpan,TimeSpan)"
names = [ "TView" ]

Based on that, I believe docfx is analyzing LogConsistentGrain<TView> and hitting the RegisterTimer method from the base Grain class.

KalleOlaviNiemitalo commented 4 years ago

TheFunc{`0,`1} parameter looks wrong. I think it should have been System.Func{System.Object,System.Threading.Tasks.Task}; the namespace is missing, too. Did the string come from here? https://github.com/dotnet/docfx/blob/3cde049d78c60026badb4825fca055d5433069a5/src/Microsoft.DocAsCode.Metadata.ManagedReference.Roslyn/Visitors/VisitorHelper.cs#L70

ReubenBond commented 4 years ago

The string is verbatim from the debugger. I can see that it originates in GetSpecId from symbol.Accept(SpecIdCoreVisitor.Instance), which returns:

Orleans.Grain.RegisterTimer(Func{`0,`1},System.Object,TimeSpan,TimeSpan)
ReubenBond commented 4 years ago

Stepping through with a debugger, I see the wrong-looking value originates from this symbol.GetDocumentationCommentId() call: https://github.com/dotnet/docfx/blob/7010df7ec2f466f6371db0dd7a5e220d9c77db53/src/Microsoft.DocAsCode.Metadata.ManagedReference.Roslyn/Visitors/VisitorHelper.cs#L70

Which returns 

M:Orleans.Grain.RegisterTimer(Func{`0,`1},System.Object,TimeSpan,TimeSpan)
KalleOlaviNiemitalo commented 4 years ago

TimeSpan is not qualified with System. either, even though Grain.cs has using System;. Perhaps the metadata build is somehow not using the correct reference assemblies, as if NuGet packages had not been restored yet.

What if you remove all the C# code but keep the build system and add something simple:

using System;

namespace Orleans
{
    public interface IDemo
    {
        void TypeResolution(TimeSpan timeSpan, Func<Type> function);
    }
}

Will that correctly get commentId: M:Orleans.IDemo.TypeResolution(System.TimeSpan,System.Func{System.Type}) in the metadata YAML file?

ReubenBond commented 4 years ago

I attempted to create a minimal repro for this but was unable. I can point you to a branch which demonstrates the docfx crash if that can help you

wouterroos commented 4 years ago

Hi,

Wondering if this is going to be solved. I'm having a similar issue with a solution that targets multiple target frameworks. The weird thing is that the issue only occurs when I target netcoreapp2.1 (net461 and netcoreapp3.1 seem to work fine).

"metadata": [
    {
      "src": [
        {
          "files": [
            "MyProject.sln"
          ],
          "src": "../MyProject"
        }
      ],
      "dest": "api/v3.0/netcoreapp21",
      "properties": {
        "TargetFramework": "netcoreapp2.1"
      },
      "disableGitFeatures": false,
      "disableDefaultFilter": false
    },
ChristianTellefsen commented 2 years ago

I got this the same error today, DocFX 2.59.2.

[22-04-26 11:12:48.851]Error:Error extracting metadata for [...my .csproj files...] System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index at System.Collections.Generic.List1[T].get_Item (System.Int32 index) [0x00009] in <533173d24dae460899d2b10975534bb0>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SpecIdHelper+<>c__DisplayClass4_0.<SpecTypeGenericParameter>b__0 (System.Text.RegularExpressions.Match match) [0x0001c] in <592d1f25abee4dafaa2d141f06705e0b>:0 at System.Text.RegularExpressions.Regex.Replace (System.Text.RegularExpressions.MatchEvaluator evaluator, System.Text.RegularExpressions.Regex regex, System.String input, System.Int32 count, System.Int32 startat) [0x00094] in <9c6e2cb7ddd8473fa420642ddcf7ce48>:0 at System.Text.RegularExpressions.Regex.Replace (System.String input, System.Text.RegularExpressions.MatchEvaluator evaluator, System.Int32 count, System.Int32 startat) [0x0000e] in <9c6e2cb7ddd8473fa420642ddcf7ce48>:0 at System.Text.RegularExpressions.Regex.Replace (System.String input, System.Text.RegularExpressions.MatchEvaluator evaluator) [0x00023] in <9c6e2cb7ddd8473fa420642ddcf7ce48>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SpecIdHelper.SpecTypeGenericParameter (System.Collections.Generic.IReadOnlyList1[T] names, System.String id) [0x00024] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SpecIdHelper.GetSpecId (Microsoft.CodeAnalysis.ISymbol symbol, System.Collections.Generic.IReadOnlyList1[T] typeGenericParameters, System.Collections.Generic.IReadOnlyList1[T] methodGenericParameters) [0x00035] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.AddInheritedMembers (Microsoft.CodeAnalysis.INamedTypeSymbol symbol, Microsoft.CodeAnalysis.INamedTypeSymbol type, System.Collections.Generic.Dictionary2[TKey,TValue] dict, System.Collections.Generic.IReadOnlyList1[T] typeParamterNames) [0x000a1] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.GenerateInheritance (Microsoft.CodeAnalysis.INamedTypeSymbol symbol, Microsoft.DocAsCode.Metadata.ManagedReference.MetadataItem item) [0x00087] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitNamedType (Microsoft.CodeAnalysis.INamedTypeSymbol symbol) [0x0000d] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.CodeAnalysis.CSharp.Symbols.PublicModel.NamedTypeSymbol.Accept[TResult] (Microsoft.CodeAnalysis.SymbolVisitor1[TResult] visitor) [0x00000] in <16676840a0c44317b3d24e39d7d53d5d>:0 at Microsoft.CodeAnalysis.CSharp.Symbols.PublicModel.Symbol.Microsoft.CodeAnalysis.ISymbol.Accept[TResult] (Microsoft.CodeAnalysis.SymbolVisitor1[TResult] visitor) [0x00000] in <16676840a0c44317b3d24e39d7d53d5d>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitDescendants[T] (System.Collections.Generic.IEnumerable1[T] children, System.Func2[T,TResult] getChildren, System.Func2[T,TResult] filter) [0x00024] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitNamespace (Microsoft.CodeAnalysis.INamespaceSymbol symbol) [0x00021] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.CodeAnalysis.CSharp.Symbols.PublicModel.NamespaceSymbol.Accept[TResult] (Microsoft.CodeAnalysis.SymbolVisitor1[TResult] visitor) [0x00000] in <16676840a0c44317b3d24e39d7d53d5d>:0 at Microsoft.CodeAnalysis.CSharp.Symbols.PublicModel.Symbol.Microsoft.CodeAnalysis.ISymbol.Accept[TResult] (Microsoft.CodeAnalysis.SymbolVisitor1[TResult] visitor) [0x00000] in <16676840a0c44317b3d24e39d7d53d5d>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitDescendants[T] (System.Collections.Generic.IEnumerable1[T] children, System.Func2[T,TResult] getChildren, System.Func2[T,TResult] filter) [0x00024] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.SymbolVisitorAdapter.VisitAssembly (Microsoft.CodeAnalysis.IAssemblySymbol symbol) [0x0009e] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.CodeAnalysis.CSharp.Symbols.PublicModel.AssemblySymbol.Accept[TResult] (Microsoft.CodeAnalysis.SymbolVisitor1[TResult] visitor) [0x00000] in <16676840a0c44317b3d24e39d7d53d5d>:0 at Microsoft.CodeAnalysis.CSharp.Symbols.PublicModel.Symbol.Microsoft.CodeAnalysis.ISymbol.Accept[TResult] (Microsoft.CodeAnalysis.SymbolVisitor1[TResult] visitor) [0x00000] in <16676840a0c44317b3d24e39d7d53d5d>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.RoslynMetadataExtractor.Extract (Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataOptions options) [0x000aa] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.RoslynIntermediateMetadataExtractor.GenerateYamlMetadata (Microsoft.CodeAnalysis.Compilation compilation, Microsoft.CodeAnalysis.IAssemblySymbol assembly, Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataOptions options) [0x00018] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.RoslynIntermediateMetadataExtractor.Extract (Microsoft.DocAsCode.Metadata.ManagedReference.IInputParameters key) [0x00022] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.RoslynSourceFileBuildController.ExtractMetadata (Microsoft.DocAsCode.Metadata.ManagedReference.IInputParameters parameters) [0x00007] in <592d1f25abee4dafaa2d141f06705e0b>:0 at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.GetMetadataFromProjectLevelCache (Microsoft.DocAsCode.Metadata.ManagedReference.IBuildController controller, Microsoft.DocAsCode.Metadata.ManagedReference.IInputParameters key) [0x000d0] in :0 at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.SaveAllMembersFromCacheAsync () [0x00806] in :0 at Microsoft.DocAsCode.Metadata.ManagedReference.ExtractMetadataWorker.ExtractMetadataAsync () [0x000c0] in :0