haf / expecto

A smooth testing lib for F#. APIs made for humans! Strong testing methodologies for everyone!
Apache License 2.0
663 stars 96 forks source link

Benchmark 0.12.1 not working #381

Closed Swoorup closed 4 years ago

Swoorup commented 4 years ago

I tested out a sample benchmark as posted in the readme with installation of Expecto.benchmark package. It is failing with the following result:

[06:28:21 ERR] performance tests.three serialisers errored in 00:00:00.0190000 <Expecto>
System.MissingMethodException: Method not found: 'System.Text.Encoding BenchmarkDotNet.Configs.IConfig.get_Encoding()'.
   at Benchmark.tests@42-1.Invoke(Unit unitVar)
   at Expecto.Impl.execTestAsync@566-1.Invoke(Unit unitVar)
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvoke[T,TResult](AsyncActivation`1 ctxt, TResult result1, FSharpFunc`2 part2) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\async.fs:line 398
MNie commented 4 years ago

@Swoorup could you please give information, what packages (versions) are you using? I try with the newest expecto (9.0.0) and expecto.benchmarkdotnet (9.0.0). It seems to work just fine.

image

Here is code I use to check the issue:

fsproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <GenerateProgramFile>false</GenerateProgramFile>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Main.fs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Expecto" Version="9.*" />
    <PackageReference Include="FSharp.Core" Version="4.*" />
    <PackageReference Include="Expecto.BenchmarkDotNet" Version="9.*" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.*" />
  </ItemGroup>
</Project>

main.fs:

module expectobenchmarktest
open Expecto

type ISerialiser =
  abstract member Serialise<'a> : 'a -> unit

type MySlowSerialiser() =
  interface ISerialiser with
    member __.Serialise _ =
      System.Threading.Thread.Sleep(30)

type FastSerialiser() =
  interface ISerialiser with
    member __.Serialise _ =
      System.Threading.Thread.Sleep(10)

type FastSerialiserAlt() =
  interface ISerialiser with
    member __.Serialise _ =
     System.Threading.Thread.Sleep(20)

type Serialisers() =
  let fast, fastAlt, slow =
    FastSerialiser() :> ISerialiser,
    FastSerialiserAlt() :> ISerialiser,
    MySlowSerialiser() :> ISerialiser

  [<Benchmark>]
  member __.FastSerialiserAlt() = fastAlt.Serialise "Hello world"

  [<Benchmark>]
  member __.SlowSerialiser() = slow.Serialise "Hello world"

  [<Benchmark(Baseline = true)>]
  member __.FastSerialiser() = fast.Serialise "Hello world"

[<Tests>]
let tests =
  testList "performance tests" [
    test "three serialisers" {
      benchmark<Serialisers> benchmarkConfig (fun _ -> null) |> ignore
    }
  ]

[<EntryPoint>]
let main argv =
    Tests.runTestsInAssembly defaultConfig argv
Swoorup commented 4 years ago

@MNie Ahh sorry, it appears to have the issue when it is installed with BenchmarkDotNet 0.12.1 They removed Encoding property in IConfig. https://github.com/dotnet/BenchmarkDotNet/blob/384d479fdeb6560527f1918b0be00967fddd01dd/src/BenchmarkDotNet/Configs/IConfig.cs#L42