search

ionide / Fornax

Scriptable static site generator using type safe F# DSL to define page templates.
MIT License
241 stars 44 forks source link

Always call 'getGeneratorContent' for posts in watch mode #98

Closed rdipardo closed 2 years ago

rdipardo commented 2 years ago

Since 0a7c6699f6f96b6813de53b6a6c80f5c6f0c200d, Generator.GeneratorEvaluator.evaluate loads a generator script from a cache:

https://github.com/ionide/Fornax/blob/0a7c6699f6f96b6813de53b6a6c80f5c6f0c200d/src/Fornax/Generator.fs#L180-L191

But reusing the post.fsx generator crashes fornax in watch mode; the siteContent variable is empty the second time around:

[23:09:34] multiple files generated in 118ms
[23:09:35] '/home/rob/dev/Fornax/src/Fornax.Template/_public/about.html' generated in 476ms
[23:09:35] '/home/rob/dev/Fornax/src/Fornax.Template/_public/contact.html' generated in 504ms
Model+SiteContents has 7 posts
[23:09:36] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post6.html' generated in 537ms
Model+SiteContents has 7 posts
[23:09:37] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post.html' generated in 504ms
Model+SiteContents has 7 posts
[23:09:38] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post2.html' generated in 505ms
Model+SiteContents has 7 posts
[23:09:39] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post4.html' generated in 520ms
Model+SiteContents has 7 posts
[23:09:40] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post5.html' generated in 533ms
Model+SiteContents has 7 posts
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/post3.html' generated in 509ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/js/sampleJsFile.js' generated in 91ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/images/avatar.jpg' generated in 96ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/images/bulma.png' generated in 95ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/images/favicon.png' generated in 89ms
[23:09:41] '/home/rob/dev/Fornax/src/Fornax.Template/_public/style/style.css' generated in 100ms
Model+SiteContents has 7 posts
[23:09:42] '/home/rob/dev/Fornax/src/Fornax.Template/_public/posts/subdir/post3.html' generated in 2ms
Generation time: 00:00:32.8188529
[23:09:42] Watch mode started. Press any key to exit.
[23:09:42 INF] Smooth! Suave listener started in 166.701ms with binding 127.0.0.1:8080

[... edit post.md ...]

[23:09:50] Changes detected: /home/rob/dev/Fornax/src/Fornax.Template/posts/post.md
[23:09:53] multiple files generated in 638ms
[23:09:53] '/home/rob/dev/Fornax/src/Fornax.Template/_public/about.html' generated in 1ms
[23:09:53] '/home/rob/dev/Fornax/src/Fornax.Template/_public/contact.html' generated in 1ms
Model+SiteContents has 0 posts
An unexpected error happend: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Collections.Generic.KeyNotFoundException: An index satisfying the predicate was not found in the collection.
   at Microsoft.FSharp.Collections.SeqModule.Find[T](FSharpFunc`2 predicate, IEnumerable`1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 677
   at FSI_0022.Post.generate'(SiteContents ctx, String page)
   at lambda_method21(Closure , Unit , SiteContents , String , String )
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Generator.EvaluatorHelpers.helper@56(Object next, FSharpList`1 args) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 66
   at Generator.EvaluatorHelpers.invokeFunction(Object f, IEnumerable`1 args) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 70
   at Generator.GeneratorEvaluator.evaluate@194-1.Invoke(Object generator) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 195
   at System.Runtime.CompilerServices.RuntimeHelpers.DispatchTailCalls(IntPtr callersRetAddrSlot, IntPtr callTarget, IntPtr retVal)
   at Generator.GeneratorEvaluator.evaluate(FsiEvaluationSession fsi, SiteContents siteContent, String generatorPath, String projectRoot, String page) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 190
   at Generator.generate(FsiEvaluationSession fsi, Config cfg, SiteContents siteContent, String projectRoot, String page) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 333
   at Generator.action@1-3(String projectRoot, FsiEvaluationSession fsi, Config config, SiteContents sc, String filePath) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 496
   at Generator.generateFolder(String projectRoot, Boolean isWatch) in /home/rob/dev/Fornax/src/Fornax/Generator.fs:line 495
   at Fornax.guardedGenerate@226(String cwd, Unit unitVar0) in /home/rob/dev/Fornax/src/Fornax/Fornax.fs:line 228
Finished (Failed) 'TestTemplate' in 00:00:43.2120491

As long as getGeneratorContent is called in the rebuild cycle, file watching works as intended:

https://user-images.githubusercontent.com/59004801/131754553-a092ccb8-65ef-449a-b3fa-d04525322821.mp4

This patch makes evaluate aware of watch mode(*) so that getGeneratorContent can be called on Markdown pages after a file change event.

Fixes #96 Closes #84 (specifically https://github.com/ionide/Fornax/issues/84#issuecomment-848240773)

(*) In practice, by propagating the value of isWatch via Generator.generateFolder -> Generator.generate -> Generator.GeneratorEvaluator.evaluate.

Because Markdown files are never handled by Generator.runOnceGenerators, there evaluate is called with isWatch hard-coded as false, so the former behaviour hasn't changed.

Generator.GeneratorEvaluator.evaluateMultiple is also unchanged because it doesn't use the cache:

https://github.com/ionide/Fornax/blob/abd3baa2db9b66af7b95cfb5f98cefb32fa8d8fa/src/Fornax/Generator.fs#L212-L215

This could mean sacrificing some of the efficiency that 0a7c6699f6f96b6813de53b6a6c80f5c6f0c200d was aiming for, but IMHO a really efficient file watcher would only rebuild the changed file, not the entire website.

Actual performance will vary: the video above was captured on a Win 8.1-era notebook with a 900 MHz AMD processor

bigjonroberts commented 2 years ago

I ran into this same issue. I'd like to test out a build from this branch, but I'm unsure if there is a quick/easy way to try it out? It looks like dotnet tool install supports a configfile setting to point to another nuget feed and adjust other settings there. Do builds from here get published somewhere as pre-releases?

rdipardo commented 2 years ago

@bigjonroberts,

I'd like to test out a build from this branch, but I'm unsure if there is a quick/easy way to try it out?

I'm not sure why, but for me the latest stable fornax doesn't crash anymore! Maybe a debug build configuration was the source of the problem all along :thinking:

rdipardo commented 2 years ago

I still don't know why the stable release works fine when development builds continue to crash.

At least I'm sure it's not the build configuration: the complete invocation of the Build and Publish targets contain property:Configuration=Release:

$ FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH=/usr/share/dotnet dotnet fake build -t TestTemplate 

The last restore is still up to date. Nothing left to do.
run TestTemplate
Building project with version: LocalBuild
Shortened DependencyGraph for Target TestTemplate:
<== TestTemplate
   <== Publish
      <== Build
         <== Restore
            <== Clean

The running order is:
Group - 1
  - Clean
Group - 2
  - Restore
Group - 3
  - Build
Group - 4
  - Publish
Group - 5
  - TestTemplate
Starting target 'Clean'
Finished (Success) 'Clean' in 00:00:00.0177219
Starting target 'Restore'
Starting task 'DotNet:restore'
> "/usr/bin/mono" --version (In: false, Out: true, Err: true)
/home/rob/repos/dotnet/Fornax> "dotnet"  msbuild /version /nologo (In: false, Out: true, Err: true)
17.1.0.7609
/home/rob/repos/dotnet/Fornax> "dotnet"  restore "" /bl:/tmp/tmpkAQDnQ.tmp.binlog (In: false, Out: false, Err: false)
/usr/share/dotnet/sdk/6.0.200/MSBuild.dll -nologo -maxcpucount -target:Restore -verbosity:m /bl:/tmp/tmpkAQDnQ.tmp.binlog ./Fornax.sln
  Determining projects to restore...
  All projects are up-to-date for restore.
Finished (Success) 'DotNet:restore' in 00:00:12.3751631
Finished (Success) 'Restore' in 00:00:12.3853960
Starting target 'Build'
Starting task 'DotNet:build'
/home/rob/repos/dotnet/Fornax> "dotnet"  build "" --configuration Release /bl:/tmp/tmp0r2MRE.tmp.binlog (In: false, Out: false, Err: false)
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

/usr/share/dotnet/sdk/6.0.200/MSBuild.dll -consoleloggerparameters:Summary -maxcpucount -property:Configuration=Release -restore -verbosity:m /bl:/tmp/tmp0r2MRE.tmp.binlog ./Fornax.sln
  Determining projects to restore...
  All projects are up-to-date for restore.
  Fornax.Core -> /home/rob/repos/dotnet/Fornax/src/Fornax.Core/bin/Release/netstandard2.0/Fornax.Core.dll
  Fornax.Core.UnitTests -> /home/rob/repos/dotnet/Fornax/test/Fornax.Core.UnitTests/bin/Release/net6.0/Fornax.Core.UnitTests.dll
  Fornax -> /home/rob/repos/dotnet/Fornax/src/Fornax/bin/Release/net5.0/Fornax.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:01:24.13
Finished (Success) 'DotNet:build' in 00:01:25.7469377
Finished (Success) 'Build' in 00:01:25.7495507
Starting target 'Publish'
Starting task 'DotNet:publish': src/Fornax
/home/rob/repos/dotnet/Fornax> "dotnet"  publish src/Fornax --configuration Release --output /home/rob/repos/dotnet/Fornax/temp /bl:/tmp/tmp8h4HMY.tmp.binlog (In: false, Out: false, Err: false)
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

/usr/share/dotnet/sdk/6.0.200/MSBuild.dll -maxcpucount -property:PublishDir=/home/rob/repos/dotnet/Fornax/temp -property:Configuration=Release -restore -target:Publish -verbosity:m /bl:/tmp/tmp8h4HMY.tmp.binlog src/Fornax/Fornax.fsproj
  Determining projects to restore...
  All projects are up-to-date for restore.
  Fornax.Core -> /home/rob/repos/dotnet/Fornax/src/Fornax.Core/bin/Release/netstandard2.0/Fornax.Core.dll
  Fornax -> /home/rob/repos/dotnet/Fornax/src/Fornax/bin/Release/net5.0/Fornax.dll
  Fornax -> /home/rob/repos/dotnet/Fornax/temp/
Finished (Success) 'DotNet:publish' in 00:00:11.5370303
Finished (Success) 'Publish' in 00:00:11.5400999
Starting target 'TestTemplate'
templateDir: /home/rob/repos/dotnet/Fornax/src/Fornax.Template/
/home/rob/repos/dotnet/Fornax/src/Fornax.Template/> "dotnet" /home/rob/repos/dotnet/Fornax/temp/Fornax.dll watch (In: false, Out: false, Err: false)
[17:41:46] multiple files generated in 610ms
[17:41:46] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/about.html' generated in 767ms
[17:41:47] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/contact.html' generated in 613ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post3.html' generated in 632ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post.html' generated in 2ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post4.html' generated in 1ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post6.html' generated in 2ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post5.html' generated in 2ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/post2.html' generated in 2ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/js/sampleJsFile.js' generated in 117ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/style/style.css' generated in 1ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/images/bulma.png' generated in 0ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/images/avatar.jpg' generated in 0ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/images/favicon.png' generated in 0ms
[17:41:48] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/posts/subdir/post3.html' generated in 1ms
Generation time: 00:00:27.6688370
[17:41:48] Watch mode started. Press any key to exit.
[17:41:48 INF] Smooth! Suave listener started in 114.535ms with binding 127.0.0.1:8080
$ echo -e "\n\n### More\n" >> src/Fornax.Template/posts/post.md 

[17:42:40] Changes detected: /home/rob/repos/dotnet/Fornax/src/Fornax.Template/posts/post.md
[17:42:42] multiple files generated in 635ms
[17:42:42] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/about.html' generated in 1ms
[17:42:42] '/home/rob/repos/dotnet/Fornax/src/Fornax.Template/_public/contact.html' generated in 0ms
An unexpected error happend: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Collections.Generic.KeyNotFoundException: An index satisfying the predicate was not found in the collection.
   at Microsoft.FSharp.Collections.SeqModule.Find[T](FSharpFunc`2 predicate, IEnumerable`1 source) in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 677
   at FSI_0022.Post.generate'(SiteContents ctx, String page)
   at lambda_method21(Closure , Unit , SiteContents , String , String )
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Generator.EvaluatorHelpers.helper@56(Object next, FSharpList`1 args) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 66
   at Generator.EvaluatorHelpers.invokeFunction(Object f, IEnumerable`1 args) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 70
   at Generator.GeneratorEvaluator.evaluate@194-1.Invoke(Object generator) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 195
   at Generator.GeneratorEvaluator.evaluate(FsiEvaluationSession fsi, SiteContents siteContent, String generatorPath, String projectRoot, String page) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 184
   at Generator.generate(FsiEvaluationSession fsi, Config cfg, SiteContents siteContent, String projectRoot, String page) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 335
   at Generator.generateFolder(String projectRoot, Boolean isWatch) in /home/rob/repos/dotnet/Fornax/src/Fornax/Generator.fs:line 495
   at Fornax.guardedGenerate@226(String cwd, Unit unitVar0) in /home/rob/repos/dotnet/Fornax/src/Fornax/Fornax.fs:line 228
Finished (Failed) 'TestTemplate' in 00:01:23.6215582
rdipardo commented 2 years ago

Closing now that bafe6193d has been merged