apple / swift-nio

Event-driven network application framework for high performance protocol servers & clients, non-blocking.
https://swiftpackageindex.com/apple/swift-nio/documentation
Apache License 2.0
7.96k stars 649 forks source link

`removeItem` performance #2933

Open pepicrft opened 2 days ago

pepicrft commented 2 days ago

👋🏼 Hi,

First of all, thanks for this amazing work.

We are using the file-system IO components at Tuist, and I'd like to bring up something that we notice regarding the performance of the NIOFileSystem.FileSystem.removeItem function.

When we run it against a large directory with many nested directories and and files, we noticed the performance is much worse than using FileManager or just rm -rf in a Linux environment. We benchmarked the performance with hyperfine and the results are below.

Looking at the implementation, it seems that we are not parallelising at all, so I was wondering if this is something intentional, or if it's ok to refactor that code to run as much as possible in parallel.

Bechmarks

With NIOFileSystem

Benchmark

With Foundation's FileManager

Benchmark:

With rm -rf

Benchmark: Time (mean ± σ): 3.150 s ± 0.386 s [User: 0.022 s, System: 1.666 s] Range (min … max): 2.225 s … 3.585 s 10 runs

FranzBusch commented 2 days ago

Thanks for filling this! This is definitely something that we would take a patch for and parallelise the removal similar to how we did for copying in https://github.com/apple/swift-nio/pull/2806.