IntelLabs / vdms

VDMS: Your Favorite Visual Data Management System
MIT License
84 stars 31 forks source link

Cache Descriptor Set Path #202

Closed ifadams closed 1 month ago

ifadams commented 1 month ago

Hey Folks,

This is a draft PR for the descriptor set path caching.

Currently, to the best of my knowledge, we can't actually delete a descriptor set after creation so this is a fairly minimal change.

It relies on the fact that a descriptor set path, as well as its dimensionality, will never change after creation. This lets us bypass the database query after the initial caching of the path and dimensions for a named set.

ifadams commented 1 month ago

Do you think it is necessary to also clear the cache when the index is deleted in

https://github.com/IntelLabs/vdms/blob/4eaa75030f45836bb9b5dc5775601148086e5a41/src/vcl/DescriptorSet.cc#L94

This is a concern of mine longer term, but it doesn't look like the "delete" is actually externally connected at the moment. i.e. I dont see any VDMS level logic that lets us fully remove an index. At most, with some gymnastics we might be able to remove the descriptor set reference from the graph database, but even thats a little murky.

@s-gobriel do you know if there's an example or call we can check to see if a client can actually trigger a delete of either the index or the referencing PMGD node?

github-actions[bot] commented 1 month ago

Target CPP Coverage: 64.4418% Source CPP Coverage: 64.3906%

Target Python Coverage: 98% Source Python Coverage: 98%

github-actions[bot] commented 1 month ago

Target CPP Coverage: 64.4418% Source CPP Coverage: 64.3906%

Target Python Coverage: 98% Source Python Coverage: 98%

github-actions[bot] commented 1 month ago

Target CPP Coverage: 64.4418% Source CPP Coverage: 64.3906%

Target Python Coverage: 98% Source Python Coverage: 98%

github-actions[bot] commented 1 month ago

Target CPP Coverage: 64.4418% Source CPP Coverage: 64.3906%

Target Python Coverage: 98% Source Python Coverage: 98%

github-actions[bot] commented 1 month ago

Target CPP Coverage: 64.4418% Source CPP Coverage: 64.3906%

Target Python Coverage: 98% Source Python Coverage: 98%

github-actions[bot] commented 1 month ago

Target CPP Coverage: 64.4418% Source CPP Coverage: 64.3906%

Target Python Coverage: 98% Source Python Coverage: 98%