pmem / ndctl

A "device memory" enabling project encompassing tools and libraries for CXL, NVDIMMs, DAX, memory tiering and other platform memory device topics.
Other
262 stars 135 forks source link

ndctl create-namespace #208

Open HBSDLJZ opened 2 years ago

HBSDLJZ commented 2 years ago

{ "regions":[ { "dev":"region1", "size":270582939648, "align":16777216, "available_size":270582939648, "max_available_extent":270582939648, "type":"pmem", "iset_id":4649103193157805124, "persistence_domain":"memory_controller" }, { "dev":"region0", "size":270582939648, "align":16777216, "available_size":270582939648, "max_available_extent":270582939648, "type":"pmem", "iset_id":4649103193157805124, "persistence_domain":"memory_controller" }, ] } ndctl create-namespace: I want to create a 512GB namespace, but I have two 256 pmem regions. How do I create them? My scenario is to create a mapping file with a size of 512GB. Thank you~

hramrach commented 2 years ago

Please see https://docs.pmem.io/ndctl-user-guide/concepts https://pmem.io/blog/2018/05/using-persistent-memory-devices-with-the-linux-device-mapper/

sscargal commented 2 years ago

@HBSDLJZ There's no hardware RAID-like features built into the CPU and Memory Controllers to allow you to aggregate capacity across CPU sockets (Regions). You can use software RAID as hramrach suggested, but this adds latency because we're forcing non-NUMA operations over the UPI. See Storage Redundancy with Intel® Optane™ Persistent Memory Modules.

For optimal performance, it's best to partition the data to fit within the NUMA local PMem namespaces and have the application access the data locally within the NUMA. If this isn't possible, check with your system vendor to see if they support a non-balanced DIMM population, ie: move all PMem to one of the sockets and ensure the app runs on that socket.

hramrach commented 2 years ago

It is not clear that these are indeed DIMMs attached to different sockets - that's something that needs to be determined by looking at the hardware and associated documentation.

It is suggested in the generic documentation that some firmware may optionally program the memory controller to interleave DIMMs that are attached to the same socket or present them as separate regions, and only knowing your hardware can give you an answer to what you really have.

HBSDLJZ commented 2 years ago

thankyou