fsprojects / FSharp.TypeProviders.SDK

The SDK for creating F# type providers
https://fsprojects.github.io/FSharp.TypeProviders.SDK/
MIT License
298 stars 94 forks source link

Provided Types performance, parsing is still slow #341

Open Thorium opened 4 years ago

Thorium commented 4 years ago

I know there has been perf-improvements recently, but I think we are not yet in a state where it'd be acceptable.

I'm trying to use SwaggerProvider (reference from NuGet taking around 2 sec), which basically reads a web-schema from internet, and then parses it with Microsoft.OpenApi.Reader (this takes around 3sec total), and then spends next 40 seconds in ProvidedTypes.fs parsing.

Repro steps

Run with: dotnet fsi /langversion:preview this script:

#r "nuget: SwaggerProvider"
let [<Literal>] Schema = "https://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.sdk.json";;
let t = System.DateTime.Now;; //#time didn't give correct results so let's user the old way...
type Stripe = SwaggerProvider.OpenApiClientProvider<Schema, PreferAsync = true>;;
printfn "%f" (System.DateTime.Now-t).TotalSeconds;;

Expected behavior

I'd expect this to be faster. If impossible, then the user should be let known that something is being done.

Actual behavior

Now everything is just freezing until suddenly 40 secs later, --tadaa--, there is a correct result!

Known workarounds

Go get some coffee.

Related information

You can test with your own machine, I had an Dell 7490 laptop (i7).

Thorium commented 4 months ago

There are a lot of nested types in this one. The time is spent on ProvidedTypes.fs on method .Compile(isHostedExecution) This takes 7 seconds wait: // phase 2 - emit member definitions This takes 45 seconds wait: // phase 3 - emit member code Most of the time in these 2: // Emit the constructor (if any) // Emit the methods

If iterateTypes function could just run parallel on nestedTypes, that could help a lot. However the parameter f is doing operations on non-theadsafe manner. So using Array.Parallel.iter would need a major refactor to avoid causing erros.