statiqdev / Statiq.Web

Statiq Web is a flexible static site generator written in .NET.
https://statiq.dev/web
Other
1.64k stars 234 forks source link

Stack overflow. Repeat 611 times: Netlify deploy #1020

Closed RohitM-IN closed 5 months ago

RohitM-IN commented 5 months ago

I found this issue when deploying i forgot to add appsettings.json and this happened after adding the file it worked like a charm

program start

    public class Program
    {
        public static async Task<int> Main(string[] args) => 
            await Bootstrapper
            .Factory
            .CreateDocs(args)
            .DeployToNetlify(
                Config.FromSetting<string>("NetlifySiteId"),
                Config.FromSetting<string>("NetlifyAccessToken")
            )
            .AddSetting(WebKeys.AdditionalSearchResultFields, new List<string> { Keys.Excerpt })
            .RunAsync();
    }

Steps to reproduce :

1) Dont Add appsettings.json 2) run deploy with args deploy -s IgnoreInvalidXrefs=true

Stack overflow.
Repeat 611 times:
--------------------------------
   at Statiq.Common.TypeHelper.ExpandValue(System.String, System.Object, Statiq.Common.IMetadata)
   at Statiq.Common.TypeHelper.TryExpandAndConvert[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.String, System.Object, Statiq.Common.IMetadata, System.__Canon ByRef)
   at Statiq.Common.IMetadataGetExtensions.TryGetValue[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Statiq.Common.IMetadata, System.String, System.__Canon ByRef)
   at Statiq.Common.IMetadataGetExtensions.Get[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Statiq.Common.IMetadata, System.String)
   at Statiq.Common.Config+<>c__DisplayClass24_0`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<FromSetting>b__0(Statiq.Common.IDocument, Statiq.Common.IExecutionContext)
   at Statiq.Common.Config`1+<GetAndTransformValueAsync>d__6[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Common.Config`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetAndTransformValueAsync(Statiq.Common.IDocument, Statiq.Common.IExecutionContext, System.Func`2<System.__Canon,System.__Canon>)
   at Statiq.Common.Config`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Statiq.Common.IConfig.GetValueAsync(Statiq.Common.IDocument, Statiq.Common.IExecutionContext)
   at Statiq.Common.IConfig.Statiq.Common.IMetadataValue.Get(System.String, Statiq.Common.IMetadata)
   at Statiq.Common.TypeHelper.ExpandValue(System.String, System.Object, Statiq.Common.IMetadata)
--------------------------------
   at Statiq.Common.TypeHelper.ExpandValue(System.String, System.Object, Statiq.Common.IMetadata)
   at Statiq.Common.TypeHelper.TryExpandAndConvert[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.String, System.Object, Statiq.Common.IMetadata, System.__Canon ByRef)
   at Statiq.Common.IMetadataGetExtensions.TryGetValue[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Statiq.Common.IMetadata, System.String, System.__Canon ByRef)
   at Statiq.Common.IMetadataGetExtensions.Get[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Statiq.Common.IMetadata, System.String, System.__Canon)
   at Statiq.Common.IMetadataConversionExtensions.GetString(Statiq.Common.IMetadata, System.String, System.String)
   at Statiq.Web.Pipelines.Deployment+<>c.<.ctor>b__0_1(Statiq.Common.IReadOnlySettings)
   at Statiq.Common.Config+<>c__DisplayClass27_0`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<FromSettings>b__0(Statiq.Common.IDocument, Statiq.Common.IExecutionContext)
   at Statiq.Common.Config`1+<GetAndTransformValueAsync>d__6[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Common.Config`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetAndTransformValueAsync(Statiq.Common.IDocument, Statiq.Common.IExecutionContext, System.Func`2<System.__Canon,System.__Canon>)
   at Statiq.Common.ConfigExtensions.GetValueAsync[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Statiq.Common.Config`1<System.__Canon>, Statiq.Common.IDocument, Statiq.Common.IExecutionContext, System.Func`2<System.__Canon,System.__Canon>)
   at Statiq.Common.ConfigModule`1+<ExecuteContextAsync>d__4[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Common.ConfigModule`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ExecuteContextAsync(Statiq.Common.IExecutionContext)
   at Statiq.Common.Module+<ExecuteAsync>d__0.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Common.Module.ExecuteAsync(Statiq.Common.IExecutionContext)
   at Statiq.Core.Engine+<ExecuteModulesAsync>d__115.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Collections.Immutable.ImmutableArray`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Collections.Immutable, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Core.Engine.ExecuteModulesAsync(Statiq.Core.ExecutionContextData, Statiq.Common.IExecutionContext, System.Collections.Generic.IEnumerable`1<Statiq.Common.IModule>, System.Collections.Immutable.ImmutableArray`1<Statiq.Common.IDocument>, Microsoft.Extensions.Logging.ILogger)
   at Statiq.Core.ExecutionContext+<ExecuteModulesAsync>d__67.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Collections.Immutable.ImmutableArray`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Collections.Immutable, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Core.ExecutionContext.ExecuteModulesAsync(System.Collections.Generic.IEnumerable`1<Statiq.Common.IModule>, System.Collections.Generic.IEnumerable`1<Statiq.Common.IDocument>)
   at Statiq.Core.ExecuteIf+<ExecuteContextAsync>d__16.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Core.ExecuteIf.ExecuteContextAsync(Statiq.Common.IExecutionContext)
   at Statiq.Common.Module+<ExecuteAsync>d__0.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Common.Module.ExecuteAsync(Statiq.Common.IExecutionContext)
   at Statiq.Core.Engine+<ExecuteModulesAsync>d__115.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Collections.Immutable.ImmutableArray`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Collections.Immutable, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Core.Engine.ExecuteModulesAsync(Statiq.Core.ExecutionContextData, Statiq.Common.IExecutionContext, System.Collections.Generic.IEnumerable`1<Statiq.Common.IModule>, System.Collections.Immutable.ImmutableArray`1<Statiq.Common.IDocument>, Microsoft.Extensions.Logging.ILogger)
   at Statiq.Core.PipelinePhase+<ExecuteAsync>d__22.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon ByRef)
   at Statiq.Core.PipelinePhase.ExecuteAsync(Statiq.Core.Engine, System.Collections.Concurrent.ConcurrentDictionary`2<System.String,Statiq.Core.PhaseResult[]>, System.Collections.Concurrent.ConcurrentDictionary`2<Statiq.Core.PipelinePhase,System.Collections.Concurrent.ConcurrentBag`1<Statiq.Core.AnalyzerResult>>)
   at Statiq.Core.Engine+<>c__DisplayClass114_0.<GetPhaseTaskAsync>b__3(System.Threading.Tasks.Task[])
   at System.Threading.Tasks.TaskFactory`1+<>c[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<ContinueWhenAllImpl>b__57_1(System.Threading.Tasks.Task`1<System.Threading.Tasks.Task[]>, System.Object)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].InnerInvoke()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
daveaglick commented 5 months ago

Interesting! Did you just add an empty appsettings.json file, or did it contain anything? It looks as though it failed in the deployment pipeline while trying to get a particular value (though it's hard to tell what value). I'm going to move this issue over to Statiq Web since that's where the deployment pipeline lives and continue investigating.

daveaglick commented 5 months ago

Quick question - where are "NetlifySiteId" and "NetlifyAccessToken" defined? I see them being used to configure the Netlify deployment, but where are those values coming from? Were they defined anywhere before you added the appsettings.json file?

daveaglick commented 5 months ago

Ah, I see what's going on here. Basically, the DeployToNetlify() extension method is intended to specify the actual value of the NetlifySiteId and NetlifyAccessToken settings (whether that's directly or via something like a delegate). However, to allow more complicated logic it does take a Config<string>. Here's the code of that extension, which might make this a little clearer:

public static TBootstrapper DeployToNetlify<TBootstrapper>(
    this TBootstrapper bootstrapper,
    Config<string> siteId,
    Config<string> accessToken)
    where TBootstrapper : IBootstrapper =>
    bootstrapper
        .AddSettingsIfNonExisting(new Dictionary<string, object>
        {
            { WebKeys.NetlifySiteId, siteId },
            { WebKeys.NetlifyAccessToken, accessToken }
        });

See how it's just calling AddSettingsIfNonExisting()? When those settings aren't defined elsewhere, the Program.cs code above ends up adding them as Config.FromSetting<string>("NetlifySiteId"), etc. So what's happening is that the Netlify deployment module looks for the setting NetlifySiteId, finds it, and then evaluates the Config.FromSetting<string>("NetlifySiteId"), which looks for the setting NetlifySiteId, finds it, etc. on and on. By adding an appsettings.json file with those values, the AddSettingsIfNonExisting() sees that they're already defined and doesn't redefine them.

The "solution" here is to either specify the value of these settings directly in the Program.cs file, or specify them elsewhere like an appsettings.json file or an environment variable. In other words, you don't need to call DeployToNetlify() in your Program.cs at all if those settings are defined somewhere else - Statiq will find them and trigger the deployment if they're available. Think of the DeployToNetlify() extension not as a "turn it on" switch but as a "configure it" helper if that makes sense.

I'll make a note in the documentation to clarify this for all the deployment pipelines, and I also made a small improvement to how this sort of recursive key expansion is reported so it's clearer. Here's what your scenario (without the appsettings.json file) would output going forward:

image