Notes on switching to .NET Core tooling in "fsprojects" repos

dsyme commented 5 years ago

Let's use this issue to document how to switch to .NET-Core based tooling in "fsprojects" repos. A good measure of this is if "mono" is no longer required in the repo at all (except maybe for testing)

.NET Core Paket

Typical commands to acquire and run paket:

dotnet tool install paket --tool-path .paket
.paket\paket restore

.NET Core FAKE

dotnet tool install fake-cli --tool-path .fake --version 5.12.6
.fake\fake run build.fsx

Switching to FAKE 5

Many "fsprojects" repos use FAKE 4. See https://fake.build/fake-migrate-to-fake-5.html

At top of build.fsx you'll need this:

#r @"paket:
source https://nuget.org/api/v2
framework netstandard2.0
nuget Fake.Core.Target
nuget Fake.Core.ReleaseNotes
nuget Fake.DotNet.Cli
nuget Fake.DotNet.AssemblyInfoFile
nuget Fake.Tools.Git //"

#if !FAKE
#load "./.fake/build.fsx/intellisense.fsx"
#r "netstandard" // Temp fix for https://github.com/fsharp/FAKE/issues/1985
#endif

Notes:

Delete build.fsx.lock whenever you update dependencies
Note the strange //" on the last line, used by FAKE to detect the Paket section in the script

Typical opens:

open Fake 
open Fake.Core.TargetOperators
open Fake.Core 
open Fake.Tools.Git
open Fake.DotNet
open Fake.IO
open Fake.IO.Globbing.Operators
open System
open System.IO

Typical code changes. When in doubt search for the old identifier in FAKE repository source

open Fake.Git    --> open Fake.Tools.Git
open Fake.ReleaseNotesHelper --> remove

parseReleaseNotes --> ReleaseNotes.parse  (open Fake.Core)
Target  --> Target.create   (open Fake.Core)
CreateFSharpAssemblyInfo --> AssemblyInfoFile.createFSharp  (open Fake.DotNet)
Attribute.Title --> AssemblyInfo.Title (open Fake.DotNet)
CleanDirs --> Shell.cleanDirs  (open Fake.IO)
!! -->   open Fake.IO.Globbing.Operators
Paket.Pack --> Paket.pack (open Fake.DotNet.Paket)
toLines --> String.toLines
fullclean --> Repository.fullclean  (open Fake.Tools.Git)
copyRecursive --> Shell.copyRecursive
stageAll --> Staging.stageAll
commit --> Commit.exec
DoNothing --> ignore
tracefn --> Trace.tracefn
RunTargetOrDefault --> Target.runOrDefault

Things with no alternative:

executeFSIWithArgs

Other notes:

Paket.pack will need an explicit ToolPath for .NET Core paket tool local install, e.g.

let paketToolPath = __SOURCE_DIRECTORY__ + (if Environment.isWindows then "\\.paket\\paket.exe" else "/.paket/paket")

Target.create "NuGet" (fun _ ->
    Paket.pack (fun p -> 
        printfn "p.ToolPath = %A" p.ToolPath
        { p with 
            ...
            ToolPath = paketToolPath
        .... })
)

Typical `build.cmd` and `build.sh`

Each repo normally has a build.cmd and build.fsx that first acquire FAKE and Paket:

build.cmd:

@echo off
dotnet tool install fake-cli --tool-path .fake --version 5.12.6

dotnet tool install paket --tool-path .paket

.paket\paket.exe restore
if errorlevel 1 (
  exit /b %errorlevel%
)

.fake\fake.exe run build.fsx %*

build.sh:

#!/bin/bash
if test "$OS" = "Windows_NT"
then
  cmd /C build.cmd
else
  dotnet tool install fake-cli --tool-path .fake --version 5.12.6
  dotnet tool install paket --tool-path .paket

  .paket/paket restore
  exit_code=$?
  if [ $exit_code -ne 0 ]; then
    exit $exit_code
  fi

 .fake/fake run build.fsx $@
fi

Generating .NET Framework binaries without having Mono installed

Use this netfx.props - updated so it doesn't need Mono: https://github.com/fsprojects/FsLexYacc/blob/2f9f06a1fe327e2c334abc6b76b60cce697b78d2/netfx.props

If your project contains .EXE tools

Projects such as FsLexYacc include binary tools in their output nuget packages, and a targets file to use these

Add a "publish" step, e.g.

dotnet public -c Release -f netcoreapp2.0 src\FsLex\fslex.fsproj
Include the binaries in the nuget package, e.g.

../src/FsLex/bin/Release/net46/publish ==> build/fslex/net46 ../src/FsLex/bin/Release/netcoreapp2.0/publish ==> build/fslex/netcoreapp2.0 ../src/FsYacc/bin/Release/net46/publish ==> build/fsyacc/net46 ../src/FsYacc/bin/Release/netcoreapp2.0/publish ==> build/fsyacc/netcoreapp2.0
Adjust the targets to start the right binary using the right runner, e.g.

    <PropertyGroup>
               .....
        <FsLexToolPath Condition="'$(FsLexToolPath)' == '' AND '$(MSBuildRuntimeType)'!='Core'">$(MSBuildThisFileDirectory)/fslex/net46</FsLexToolPath>
        <FsLexToolExe Condition="'$(FsLexToolExe)' == '' AND '$(MSBuildRuntimeType)'!='Core'">fslex.exe</FsLexToolExe>
        <FsLexYaccToolRunner Condition="'$(UseFsLexYaccToolRunner)' != 'false' AND '$(FsLexYaccToolRunner)' == '' AND '$(MSBuildRuntimeType)'!='Core' AND '$(OS)' == 'Unix'">mono </FsLexYaccToolRunner>

        <FsLexToolPath Condition="'$(FsLexToolPath)' == '' AND '$(MSBuildRuntimeType)'=='Core'">$(MSBuildThisFileDirectory)/fslex/netcoreapp2.0</FsLexToolPath>
        <FsLexToolExe Condition="'$(FsLexToolExe)' == '' AND '$(MSBuildRuntimeType)'=='Core'">fslex.dll</FsLexToolExe>
        <FsLexYaccToolRunner Condition="'$(UseFsLexYaccToolRunner)' != 'false' AND '$(FsLexYaccToolRunner)' == '' AND '$(MSBuildRuntimeType)'=='Core'">dotnet </FsLexYaccToolRunner>
    </PropertyGroup>

...
        <Exec Command="$(FsLexYaccToolRunner)&quot;$(FsLexToolPath)\$(FsLexToolExe)&quot;  -o &quot;$(FsLexOutputFolder)%(FsLex.Filename).fs&quot; %(FsLex.OtherFlags) %(FsLex.Identity)"/>

Documentation generation

Problems

At time of writing FSharp.Formatting is not yet able to produce docs for netstandard2.0 DLLs. Therefore the only option for producing docs is to make net46 or similar DLLs as part of your build.
At time of writing, both the fsformatting.exe tool (normally driven by FAKE FSFormatting module) and the FSharp.Formatting libraries (normally driven by a generate.fsx script) are .NET Framework only. This means documentation generation must run on NET Framework.

Recommendations:

switch to docsrc and docs for github pages, e.g. see the revised ProjectScaffold
Get rid of generate.fsx and move to structure similar to ProjectScaffold doc generation
If still using generate.fsx then make sure generate.fsx is not a FAKE script and only generate docs using Windows. If necessary manually generate docs using fsi docsrc\generate.fsx rather than doing it from FAKE 5

Testing

TBD

Building `net4x` libraries

TBD

Examples:

https://github.com/fsprojects/FsLexYacc/pull/92

dsyme commented 5 years ago

A quick update: I managed to remove mono from FsLexYacc, though documentation generation still requires the generation of .NET 4.x binaries.

baronfel commented 5 years ago

Came across this project today that might be a way forward with documentation generation: https://twitter.com/dbrattli/status/1119488973133377536?s=19

Integrating with Sphinx might be easier overall than maintaining our own.

dsyme commented 2 years ago

Closing old issue :)

fsprojects / .github