AXL originally copied the source file to a temporary file having an _AXL extension and then renamed the file to its final destination name during AXL_Wait when complete. However, the rename operation seems to not scale well on all parallel file systems. On GPFS, writing 256 files from 64 compute nodes takes about 6 seconds, and the cost increases linearly with the node count. Presumably, when all destination files are in the same directory, the compute nodes may be thrashing for a single directory lock.
This PR disables the temporary file name so that AXL writes the file directly to the final destination name. The temporary file name can be enabled by setting AXL_USE_EXTENSION=1 or setting USE_EXTENSION=1 in AXL_Config.
AXL originally copied the source file to a temporary file having an
_AXLextension and then renamed the file to its final destination name duringAXL_Waitwhen complete. However, therenameoperation seems to not scale well on all parallel file systems. On GPFS, writing 256 files from 64 compute nodes takes about 6 seconds, and the cost increases linearly with the node count. Presumably, when all destination files are in the same directory, the compute nodes may be thrashing for a single directory lock.This PR disables the temporary file name so that AXL writes the file directly to the final destination name. The temporary file name can be enabled by setting
AXL_USE_EXTENSION=1or settingUSE_EXTENSION=1inAXL_Config.