Capitains/Nautilus (which powers scaife-viewer/scaife-cts-api) does not yet support MyCapytain 3.x (Capitains/Nautilus#87, scaife-viewer/scaife-viewer#454). I ran some smoke tests with both the API and local resolvers, and with only a few code conditionals, we seem to have similar behavior.
Improve local resolver performance
The __getitem__ and __contains__ methods make use of the descendants property which requires looping through every possible descendant of collection.
Experimenting on a fork (jacobwegner/MyCapytain), I found that refactoring resolver parsing to eschew descendants lookups to drastically improve performance. I plan on opening further discussions up with the MyCapytain dev team to see if we can land changes back to the local resolver from the upstream repo.
Support a cached version of the resolver data structure
scaife-viewer/scaife-cts-api has several layers of precomputation and caching that I think the local resolver could benefit from, so I've introduced a very basic cache into the resolver's parse method.
We may revisit finer grained caching (e.g. per repo/corpus) in the future.
jacobwegner
commented
1 year ago
What is in this PR?
Capitains/Nautilus (which powers scaife-viewer/scaife-cts-api) does not yet support MyCapytain 3.x (Capitains/Nautilus#87, scaife-viewer/scaife-viewer#454). I ran some smoke tests with both the API and local resolvers, and with only a few code conditionals, we seem to have similar behavior.
The
__getitem__and__contains__methods make use of thedescendantsproperty which requires looping through every possible descendant of collection.Experimenting on a fork (jacobwegner/MyCapytain), I found that refactoring resolver parsing to eschew
descendantslookups to drastically improve performance. I plan on opening further discussions up with the MyCapytain dev team to see if we can land changes back to the local resolver from the upstream repo.scaife-viewer/scaife-cts-api has several layers of precomputation and caching that I think the local resolver could benefit from, so I've introduced a very basic cache into the resolver's
parsemethod.We may revisit finer grained caching (e.g. per repo/corpus) in the future.