search

fjoppe / Legivel

F# Yaml 1.2 parser
https://fjoppe.github.io/Legivel
The Unlicense
60 stars 5 forks source link

APIs.guru schema parse tests #4

Closed sergey-tihon closed 6 years ago

sergey-tihon commented 6 years ago

PR contains 2 commits

  1. General dependencies cleanup
  2. Added about 656 parse tests of real-world Swagger 2.0 schemas listed here https://api.apis.guru/v2/list.json

Parsing looks very slow. 😞 @fjoppe Could you please take a look?

sergey-tihon commented 6 years ago

Legivel was able to parse only 98 out of 656 files on AppVeyour in 1h.

In the same time, it took me about 12 min to parse all 656 schemas in YAML format using YamlDotNet, parse 656 schemas in JSON format using FSharp.Data and then analyze all 2x656 parsed schemas and convert them to SwaggerSchema object model.

So we may assume that parse time of all 656 schemas using YamlDotNet is less than 6 min.

fjoppe commented 6 years ago

Thanks for testing, reults are tough, I'l look into it. Although I have yet no clue how to make it faster. I'll first focus on correctness.

sergey-tihon commented 6 years ago

I understand, but in order to be useful for SwaggerProvider, FSharp.Configuration and other projects, the parser should have reasonable performance.

fjoppe commented 6 years ago

This PR has been merged into a new branch, status: Work in Progress.

fjoppe commented 6 years ago

So, discovered the profiling feature in VS2017 last month and things finally are getting going.

This build shows 1492 tests done in 2952 seconds (of which 47 failed -all API guru tests). This makes up for a mean of 1.98 seconds per yaml file.

If you consider there is one, wich takes about 40 seconds to parse, with about 23000 lines of Yaml. And the first case I worked on, I've improved parsing for 1:05 to about 1 second.

Making progress here...

Note: The linked build is not published on nuget yet.