search

RepreZen / KaiZen-OpenAPI-Editor

Eclipse Editor for the Swagger-OpenAPI Description Language
Eclipse Public License 1.0
115 stars 12 forks source link

[ZEN-4172] Slow Performance, Out of Memory/GC resolving references on large Swagger project #446

Closed ghillairet closed 6 years ago

ghillairet commented 6 years ago

This PR optimises the validation of references by avoiding running the validation on each of them, but rather by collecting references in a document inside a Set from which validation will be performed. With avoid like that running validation on similar references more than once. It also reduces the number of call to JSON schema validator for validation of references types by caching sub schemas and making groups of identical types on which only a single validation is performed.

@tedepstein @andylowry This PR should improve performance on large specs quite a bit. What more could be done is to avoid performing expensive validation (such as type validation) on each document changes (maybe only on save). The validation is currently performed in a separate job, maybe their would be a way to cancel that job when another validation is initiated.

tedepstein commented 6 years ago

@ghillairet , thanks.

What more could be done is to avoid performing expensive validation (such as type validation) on each document changes (maybe only on save).

Discussed internally here. Summary is that we would like to optimize the delays between edit, validate, and (in API Studio) live view refresh, and would like to make these configurable so they don't necessarily have to happen on each edit. However, this is going to require some coordinated analysis & discussion, because the code that does this is very fragile right now.

The validation is currently performed in a separate job, maybe their would be a way to cancel that job when another validation is initiated.

Also discussed in that context. Comment from @tfesenko :

I started working on canceling validation jobs in https://github.com/RepreZen/KaiZen-OpenAPI-Editor/pull/273 But then I reverted it in https://github.com/RepreZen/KaiZen-OpenAPI-Editor/pull/275 because it caused the inconsistency between the editor state and live view state. We can revisit this work focusing on the valid state of the live views and error markers.

andylowry commented 6 years ago

@ghillairet @tedepstein Code review passed, and so did QA. Only thing is that I was unable to reproduce the slow performance. I tested old code both in a installed build and running from my dev environment, using the "CircularSchemas" model that Ted posted. In neither case did I see any suggestion that the editor wasn't keeping up with whatever I did.

I did see a "busy" cursor for a split second after a short delay when I stopped making changes, but that went away when I closed live views, so presumably that's nothing to do with the editor.

Testing of new version was only possible when running in dev environment, and there was no detectable difference.

I'll merge this, since it the code changes do in fact suggest significant improvements, and in QA things appear to check out functionally. But I'd be happier if I could reproduce the issue.