search

StanfordLegion / legion

The Legion Parallel Programming System
https://legion.stanford.edu
Apache License 2.0
692 stars 144 forks source link

Realm: Add recycling of sparsity maps #1192

Open lightsighter opened 2 years ago

lightsighter commented 2 years ago

Currently Realm's sparsity maps are leaked when they are destroyed and the IDs cannot be recycled. This needs to be fixed if we're regularly going to be creating and destroying sparsity maps, e.g. in the case where we are frequently calling image/preimage for gather/scatter copy operations in Legion in order to avoid running out of sparsity map IDs. It will also guarantee that the memory associated with sparsity maps is cleaned up.

lightsighter commented 2 years ago

Adding @streichler and @alexaiken for discussion. I'll continue to be the owner of this issue for now since I'll likely be the one that ends up implementing it in Realm.

apryakhin commented 1 year ago

PR to recycle sparse maps in realm has been out for review:

Depending on the priority we need to get that tested by legion team, reviewed and merged.

apryakhin commented 2 months ago

@lightsighter We won't be making it for 24.09, so it will have to happen in the next release.

lightsighter commented 2 months ago

Yes, I agree it's not going to make this release. We'll get it in the next one.

apryakhin commented 2 months ago

@reazulhoque

apryakhin commented 1 month ago

@lightsighter Just to follow up on this. We have the branch with realm changes that addresses the out-of-order reference counting. I think there is still some work on the Legion side that needs to happen. @reazulhoque for more context. It may so happen that legion priorities have shifted for the upcoming release? If yes just make sure we have it here in wording and have a consensus on when we want this to be done.

lightsighter commented 1 month ago

I think we still want to get this merged by the end of the year. Probably not before the December release but right after it. @reazulhoque has something more important that he's doing now, but coming back to clean this up will be his next priority.