search

fsprojects / FSharp.Formatting

F# tools for generating documentation (Markdown processor and F# code formatter)
https://fsprojects.github.io/FSharp.Formatting/
Other
465 stars 156 forks source link

Remove Razor doc template, replace by MiniScaffold/Waypoint/DotLiquid techniques #523

Closed dsyme closed 4 years ago

dsyme commented 4 years ago

FSharp.Formatting is a tool that takes four input formats (snippets, markdown, literate scripts, F#/.NET asssemblies) and produces two output formats (latex and html). Razor templating and a command line tool is added on top of this.

At the moment the package is split into multiple sub-packages along the lines of the input formats. This fractures the implementation, so yet another package (FSharp.Formatting.Common) is needed.

In practice FSharp.Formatting is built, tested and used as a single technology (with the exception of the Razor support which people may avoid). Using the technology is somewhat confusing because of the multiple bits involved.

Given this, I believe the split into multiple assemblies and packages is a waste of engineering effort (maintaining multiple assemblies and packages is not simple). We should have one package (FSharp.Formatting) and that's all.

This comes up in the context of #521, where the implementation would be simpler of everything was just one component

dsyme commented 4 years ago

At the moment the package is split into multiple sub-packages along the lines of the input formats. This fractures the implementation, so yet another package (FSharp.Formatting.Common) is needed.

I was wrong, there is actually only one pacakge including everything.

This makes this less important (it's the maintenance of multiple packages I find most annoying)

dsyme commented 4 years ago

@eiriktsarpalis @matthid Do you know of any reason why we make FSharp.Literate and FSharp.Formatting.Razor available as separate packages?

eiriktsarpalis commented 4 years ago

fssnip uses FSharp.Literate only.

dsyme commented 4 years ago

Hmmm would it matter if the dependency was FSharp.Formatting?

dsyme commented 4 years ago

Looking at things I think it wouldn't matter.

thinkbeforecoding commented 4 years ago

FSharp.Literate was split when ported to netstandard2.0 when FSharp.Formatting could not be ported due to Razor. As long as using FSharp.Formatting as a library is available, it's ok to stop FSharp.Literate.

http://thinkbeforecoding.com

On Fri, Jul 3, 2020 at 1:30 PM Don Syme notifications@github.com wrote:

Looking at things I think it wouldn't matter.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/fsprojects/FSharp.Formatting/issues/523#issuecomment-653501925, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABD2HJYYLTHDIQHIOND7NTRZW6L7ANCNFSM4ONNMHXA .

dsyme commented 4 years ago

Thanks, yes, that appears to be the case

eiriktsarpalis commented 4 years ago

I'm mostly put off by the fact that it has the RazorEngine.NetCore dependency. That particular library depends on aspnetcore 2.x shared framework components, that I expect might not work in future .NET releases. Ideally the razor dependency should be replaced before the two packages are merged.

eiriktsarpalis commented 4 years ago

FSharp.Formatting is more of a glue package bundling FSharp.Literate, FSharp.Formatting.Razor and all their transitive dependencies for ease of consumption in F# scripts. The inclusion of #r nuget: in F# 5.0 will probably render it obsolete.

If anything, I'd be in favour of eventually deprecating FSharp.Formatting, at least in its current shape.

dsyme commented 4 years ago

I'm mostly put off by the fact that it has the RazorEngine.NetCore dependency. That particular library depends on aspnetcore 2.x shared framework components, that I expect might not work in future .NET releases. Ideally the razor dependency should be replaced before the two packages are merged.

Yes, I understand that concern, though FSharp.Formatting appears basically usable without Razor.

The Razor dependencies appear to be .NET Standard 2.0 packages - is there a concern that these will stop working? It doesn't feel like we're picking up any shared framework dependencies any more? Or how could I tell if we are?

    <PackageReference Include="RazorEngine.NetCore" Version="[2.2.6, 3.0)" />
    <PackageReference Include="Microsoft.AspNetCore.Razor" Version="[2.2, 3.0)" />
    <PackageReference Include="Microsoft.AspNetCore.Razor.Runtime" Version="[2.2, 3.0)" />
    <PackageReference Include="Microsoft.AspNetCore.Razor.Language" Version="[2.2, 3.0)" />
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="[3.4, 4.0)" />
eiriktsarpalis commented 4 years ago

Based on what I've been told, the netstandard2.0 aspnetcore packages depend on .NET Core 2.x shared framework components, which is why they were discontinued in .NET Core 3.0 (aspnetcore is now bundled as a shared framework). I don't know whether this impacts the code paths used by FSharp.Formatting.Razor.

dsyme commented 4 years ago

I see

eiriktsarpalis commented 4 years ago

FWIW fssnip uses DotLiquid for templating. It looks like a well-maintained, zero-dependency alternative. It might be worth considering as a potential replacement in future releases.

matthid commented 4 years ago

I feel like part of this has been discussed here And the linked issues. I still think that is a good path forward. This suggestion here feels like a step back?

dsyme commented 4 years ago

Yeah, I was under the impression we'd somehow broken free of the shared dependencies and Razor wasn't an issue now we're working on .NET COre, but we haven't. Ugh

Anyway we can keep the FSharp.Literate package, it's ok.

dsyme commented 4 years ago

BTW in v5.0 I will rename

FSharp.Literate --> FSharp.Formatting.Literate
FSharp.Markdown --> FSharp.Formatting.Markdown
FSharp.CodeFormat --> FSharp.Formatting.CodeFormat

We should never have been occupying so many of these 2nd tier namespaces with what is really one technology.

dsyme commented 4 years ago

I will close this for now. If someone could help us break free of Razor for the assembly generation docs it would really great. I don't really care how much damage is done along the way as long as assembly docs come out the other end.

This would allow us to have one package FSharp.Formatting plain and simple. We have to simplify this business if we're to maintain it going forward

Krzysztof-Cieslak commented 4 years ago

If someone could help us break free of Razor for the assembly generation docs it would really great

Well, both @TheAngryByrd's MiniScaffold and Ionide's Waypoint are not using Razor for API reference generation, and those are probably two most popular solution scaffolds at the moment

dsyme commented 4 years ago

If someone could help us break free of Razor for the assembly generation docs it would really great

Well, both @TheAngryByrd's MiniScaffold and Ionide's Waypoint are not using Razor for API reference generation, and those are probably two most popular solution scaffolds at the moment

Yes, it would be great if someone could just come in and nuke the API doc generation in this repo and replace it with something half decent not depending on Razor.

The literate programming is the part of this repo I really want to save (e.g. converting scripts to notebooks, html, latex etc.). But it seems right that this repo should also have API doc generation - they make a reasonable combination rather than trying to use two systems.

Do you think we could just remove the doc generation in this repo and replace it with the WayPoint doc generation? Merge the efforts?

Krzysztof-Cieslak commented 4 years ago

Jimmy can tell us details about his solution, but as far as I understand both Waypoint and MiniScaffold depends on MetadataFormat API from this repo.

Waypoint additionally depends on FSharp.Literate for XML comment -> nice HTML transformation

dsyme commented 4 years ago

Jimmy can tell us details about his solution, but as far as I understand both Waypoint and MiniScaffold depends on MetadataFormat API from this repo.

Right, exactly, it's just the templating part for the API docs that's garbage here and causing the complexity nightmare with Razor. It's not even a lot of code.

So I'd like to make it plan of record replace the templating here used for doc generation and put in whatever you're using for Waypoint and/or MiniScaffold and iterate/improve that here. So then the "fsformatting" tool just works out of the box (and we drop the Razor support in this repo). You will still be able to use the MatadataFormat API if you want.

This is what I will plan to use for DiffSharp. I will look into making this change later.

I will be working hard to make the DiffSharp docs look really good, so I hope to make the defaults work well.

dsyme commented 4 years ago

Waypoint additionally depends on FSharp.Literate for XML comment -> nice HTML transformation

Just to mention that WriteHtml overload will be renamed to ToHtml in v5.0.

dsyme commented 4 years ago

For those following this discussion, the main question is what to do about the Razor-dependent API doc generation templating in this repo going forward.

It seems there are currently three suer-facing API doc generation solutions around, which use some of the functionality in this repo but not all (there may be more I just don't know about yet). Notes to myself:

Current FSharp.Formatting.RazorMetadataFormat.Generate

MiniScaffold

Waypoint

Seems like options for this repo are

  1. Rip out the use of Razor and offer no actual API doc generation. Let other solutions like Fornax/Waypoint and MiniScaffold blossom.

  2. Use DotLiquid for HTML templating.

  3. Offer HTML generation without such sophisticated templating

dsyme commented 4 years ago

OK I've finally decided the way forward here.

First, there will be no sophisticated templating customization in FSharp.Formatting.ApiDocs (previously called FSharp.Formatting.MetadataFormat). So there will be no dependency on any templating engine. I've decided that any dependency on any templating engine for such a foundational component is a nightmare and basically unnecessary, also getting our layering wrong.

Instead we will:

  1. Continue to provide and improve the ApiDocs model so tools like Waypoint and MiniScaffold can continue to do high quality doc generation

  2. We will bake in a default generation of ApiDocs ApiDocs.GenerateHtml that does a reasonable job of generating quality HTML docs. The implementation will be similar to Waypoint and I'll work with @Krzysztof-Cieslak on trying to get these aligned and getting searchable docs generated. There will be no customization of these docs except via an outer template (with a simple substitution of {document} and properties), plus CSS styling.

Also the ApiDocsModel (previously called GeneratorOutput provided by FSharp.Formatting.ApiDocs needs a lot of work. There are undefined type inference variables showing, for example.

Krzysztof-Cieslak commented 4 years ago

TBF, I believe that F#.Formatting shouldn't generate documentation on its own at all - we should instead work on making current Waypoint documentation template standalone (i.e. easily usable with just Fornax, without Waypoint solution scaffold) where ideal workflow would be:

  1. dotnet tool install fornax
  2. dotnet fornax new --docs that would generate current documentation template (using FSharp.Formatting.ApiDocs)
  3. Edit some config .fsx file with paths to projects/solution name/etc
  4. dotnet fornax build/watch

I'd rather have F#.Formatting focus on being a good library (i.e. providing good ApiDocs) rather than attempting to reimplement poorer version of what other solutions already provide.

EDIT: This comment is about ApiDocs generation part, FSharp.Formatting.Literate looks good to me. I think the CLI tool provided by F#.Formatting should only handle Literate programming part.

dsyme commented 4 years ago

My comments from chat with @Krzysztof-Cieslak below

The problem is that I can't cope with a two or three layer dependency in this part of the toolchain. I feel like I need a one-stop shop, partly for simplicity as a user, and partly for simplicity in iteratively making improvements (the API docs model coming out of FSHarp.Formatting needs a huge amount of work, it's just not the basic for good lookin gsignatures etc.)

Hard to explain, but I'm not too thrilled by using customizable .fsx scripted site generation for project docs. I guess I want a single-shop doc generation solution that's entirely content based apart from an outer template. Nothing else checked in, no generator scripts, nothing. But yes, there's definite overlap between static site generation and project docs, I can see the issue in overlap for sure.

So my current holding pattern is to get a Razor-free static doc generation working in FSharp.Formatting, good enough to use for DiffSharp, then focus on improvements to the Api Docs model layer to get that looking really good. That puts me in a stable position and has some advantages for the whole ecosystem - both Waypoint and MiniScaffolf consume the ApiDocModel (prev called GeneratorOutput). And we needed to get rid of that Razor dependency really badly, so that's progress

SO right now my plan is:

However I can see I'm going to be tempted to put things like the Lunr search index generation into here.

Krzysztof-Cieslak commented 4 years ago

However I can see I'm going to be tempted to put things like the Lunr search index generation into here

Index generation itself is fairly straightforward - it's basically a list of type Entry = { uri: string; title: string; content: string }, with each entry representing single page. However, it requires some wiring-up on the client (web page) site - some js, some css, search box etc.

dsyme commented 4 years ago

Index generation itself is fairly straightforward - it's basically a list of type Entry = { uri: string; title: string; content: string }, with each entry representing single page. However, it requires some wiring-up on the client (web page) site - some js, some css, search box etc.

Yup, I was looking over the SampleWaypoint code for this. It seems reasonable to integrate in here into a "basic" story without things getting too out of hand.

dsyme commented 4 years ago

I'll close this issue since Razor has now been nuked. We can discuss in further issues

dsyme commented 4 years ago

@Krzysztof-Cieslak Just to say I guess https://github.com/fsprojects/FSharp.Formatting/pull/528 takes things a step further.

It's far from perfect (and has a perf issue cracking project files) but it is like the entry level simplicity of project doc generation

dsyme commented 4 years ago

OK I'm really closing this one now, the work is most definitely done.

FSharp.Formatting 5.0.4 has been published and the documentation updated.