getavalon / core

The safe post-production pipeline - https://getavalon.github.io/2.0
MIT License
213 stars 48 forks source link

Performance optimizations using Mongo Aggregation #562

Open BigRoy opened 3 years ago

BigRoy commented 3 years ago

Issue

After #555 is merged we have access to the pymongo aggregation pipeline in avalon.io.

I'm opening this issue to remind ourselves of potentially other areas of performance improvements, for example. the recursive query for io.parenthood could be done as a single aggregation query - which is frequently used in calls that are often repeatedly done like updating loaded containers.

It's also likely that other areas that have been mentioned to be slow, e.g. #551 / #561 can see even further improvements than those PRs using aggregation. Or even new features greatly benefit from it, e.g. the recursive Input/Output relationships in #395. Similarly it could be of interest for hierarchical data queries like discussed in #410.