Delete the residual file of fsdax

Intel-bigdata / Spark-PMoF

Spark Shuffle Optimization with RDMA+AEP

Apache License 2.0

30 stars 22 forks source link

Delete the residual file of fsdax #93

Closed kelvin-qin closed 3 years ago

kelvin-qin commented 3 years ago

see Enable PMoF run in fsdax mode It seems that there is a new problem. After the Job is finished, the shuffle file is not automatically deleted. But pmempool info --stats <file> finds that the utilization rate is almost zero. Need to add a file delete operation in fsdax mode.

Tips: When using fsdax mode, you can adjust the number of executors more freely, and the program may run faster.

kelvin-qin commented 3 years ago

see #94

kelvin-qin commented 3 years ago

update!