gchq / Gaffer

A large-scale entity and relation database supporting aggregation of properties
Apache License 2.0
1.77k stars 350 forks source link

GafferPop should 'squash' GetElements Operations #3277

Closed p29876 closed 3 weeks ago

p29876 commented 1 month ago

At the moment if you run a query like g.V('1').outE(), for every edge on the vertex '1' GafferPop will run a GetElements operation with only one seed. This can be very problematic for queries that touch a lot of data as each operation has overhead and gets logged so could end up with thousands of operations being called at once.

To fix this we might need to implement custom vertex/edge traversal strategies so that all the seeds are acquired first then added to one big operation rather than lots of smaller ones.