Some questions about custom class

[cliuquan]

QUESTION: Some questions about custom class

Details

Issue Step:

1. Using pmemobj_create to create a 3GB size objpool.
2. Using pmemobj_ctl_set(pop, "heap.alloc_class.new.desc", class) to register 10 custom classes, size includes 100,200,300 byte.
3. Random to free memory from above custom class with below APIs, for example to malloc 1000 custom classes and each class size is 200B, then to free 800 classes of them randomly. pmemobj_tx_begin pmemobj_tx_xalloc pmemobj_tx_free pmemobj_tx_end
4. kill the process during above these malloc and free operations,
5. using pmemobj_open to open the objpool, repeat the step2 ~ step4
6. Finaly, only has 2.5GB of PMEM space, if the malloc not from custom class, PMDK will report there not enough space for malloc(currently have 2.2GB of fragment space).

From above steps, we can see that chunk free space is about 200k during repeat malloc and free the custom class. We think a chunk size in PMDK is 256k, so to open objpool will trigger to process undo log and earlier reclaim of zone, after the open then to register new class, meanwhile malloc a new 256k chunk for this class, instead of previous 200k old chunk.

Questions

1. Is this method current or not of register custom class, does any guidance documents or API can share?
2. for this method, the information of class can persistent when using pmemobj_ctl_set to register? Do we need to register again after open?
3. Why not to reallow the new chunk after re-register the class?
4. We also found a workaround but not sure it has other impacts, please give the comments, steps as following: using pmemobj_open to open the objpool #trigger the undo log using pmemobj_close to close the objpool then to open the objpool again #not need undo operate and zone reclaim

test log:

[cliuquan]

pmdk version: 1.12.1-rc1.

This Issue may be related to this commit b1d9a57f0277f530b18ff685dca1189b2086b111, which fix bug to ensure zones are reclaimed prior to free.

thanks.

[janekmi]

Hi. We currently operate on very limited resources. Can you give me some information on how important this issue is to you and the project you are working on right now? Thanks.

Please let me know if the information I am asking for is confidential. We can sort it out outside of the GitHub.

[cliuquan]

Hi. We use PMDK in our storage system, which is used for the company's cloud services and stores a large amount of user data. This issue will cause our storage system reporting insufficient storage space, causing the storage system to be unavailable and interrupting user services.

[janekmi]

I'm on it.

[janekmi]

Is this method current or not of register custom class, does any guidance documents or API can share?

Yes. It is still supported. Note that it is still an experimental API.

The documentation for the CTL PMEMOBJ APIs is available here: https://github.com/pmem/pmdk/blob/master/doc/libpmemobj/pmemobj_ctl_get.3.md. Specifically, you can find details explaining the meaning of some structure fields in the related header: https://github.com/pmem/pmdk/blob/master/src/include/libpmemobj/ctl.h.

Ref: https://pmem.io/pmdk/manpages/linux/master/pmem_ctl/pmem_ctl.5/

[janekmi]

for this method, the information of class can persistent when using pmemobj_ctl_set to register? Do we need to register again after open?

The allocation classes are part of the runtime state of the library and must be created after every open.

[janekmi]

From above steps, we can see that chunk free space is about 200k during repeat malloc and free the custom class. We think a chunk size in PMDK is 256k

The values between 0-127 are reserved for the default allocation classes of the library and can be used only for reading.

The recommended method for retrieving information about all allocation classes is to call this entry point for all class ids between 0 and 254 and discard those results for which the function returns an error. For reading, function returns 0 if successful, if the allocation class does not exist it sets the errno to ENOENT and returns -1;

[janekmi]

Hi @cliuquan. I hope you managed to find all the answers you needed. Please re-open if necessary.