Since there can be exceptions or failure in the final free code that happens once a transaction has called commit, there is code now in the concurrency branch to fix the DRAM states and make them consistent with the PM state as it will be after rollbacks. It is not clear it is worth introducing so much complexity for caching code. So here is a capture of solutions suggested by Philip during code review and II seems like it could work well:
As I understand it, the problem isn’t with maintaining the PM list of chunks, but with maintaining the DRAM list. (If that’s wrong, you can ignore the rest of this message.)
I. My first suggestion is that you simply let the DRAM list get out of sync, in the rare case* where there is an exception during commit. So, what are the issues that can arise when it is out of sync:
a) A chunk in the DRAM list has no free slots when you go to allocate from it. This currently asserts, I believe, because currently that is never supposed to happen. It should be no problem to instead just remove the chunk from the DRAM list and go to the next chunk.
b) There are no chunks in the DRAM list even though there are some chunks with free space in the PM list. To work around this, simply rebuild the DRAM chunk list from the PM list when it is empty. If there are no free chunks in the PM list either, then of course it will proceed to allocate another chunk.
c) last_chunk_scanned or an entry in the DRAM list points to a chunk that is no longer in the PM list. This is a problem that must be avoided. I’d have to think some more about how that might be done. Perhaps this is a showstopper for this approach.
II. Alternatively, you could simply blow away all the DRAM lists anytime an exception occurs in commit, and have them automatically rebuilt the next time a chunk is needed. (I believe the code to rebuild the lists as needed is there already, since it is the usual process whenever an existing graph is opened, isn’t it?)
luisremis
commented
5 years ago
Since there can be exceptions or failure in the final free code that happens once a transaction has called commit, there is code now in the concurrency branch to fix the DRAM states and make them consistent with the PM state as it will be after rollbacks. It is not clear it is worth introducing so much complexity for caching code. So here is a capture of solutions suggested by Philip during code review and II seems like it could work well:
As I understand it, the problem isn’t with maintaining the PM list of chunks, but with maintaining the DRAM list. (If that’s wrong, you can ignore the rest of this message.)
I. My first suggestion is that you simply let the DRAM list get out of sync, in the rare case* where there is an exception during commit. So, what are the issues that can arise when it is out of sync: a) A chunk in the DRAM list has no free slots when you go to allocate from it. This currently asserts, I believe, because currently that is never supposed to happen. It should be no problem to instead just remove the chunk from the DRAM list and go to the next chunk. b) There are no chunks in the DRAM list even though there are some chunks with free space in the PM list. To work around this, simply rebuild the DRAM chunk list from the PM list when it is empty. If there are no free chunks in the PM list either, then of course it will proceed to allocate another chunk. c) last_chunk_scanned or an entry in the DRAM list points to a chunk that is no longer in the PM list. This is a problem that must be avoided. I’d have to think some more about how that might be done. Perhaps this is a showstopper for this approach.
II. Alternatively, you could simply blow away all the DRAM lists anytime an exception occurs in commit, and have them automatically rebuilt the next time a chunk is needed. (I believe the code to rebuild the lists as needed is there already, since it is the usual process whenever an existing graph is opened, isn’t it?)
Philip