search

martijnvanbrummelen / nwipe

nwipe secure disk eraser
GNU General Public License v2.0
702 stars 79 forks source link

Wipe thread optimizations - Larger block sizes #306

Open PartialVolume opened 3 years ago

PartialVolume commented 3 years ago

Proposal: Investigate effect of increasing block size on disc I/O throughput and CPU load. Specifically on low end hardware, Pentium 4 or less powerful systems. Initial tests indicate a multi disc wipe may have reduced I/O on a single core processor due to CPU approaching 100% load on low power atom processors.

If there is a definite improvement then add a new nwipe option --blocksize=16k being 16,384 bytes. Etc.

ZigZagJoe commented 3 years ago

Backstory: This issue originated from https://github.com/PartialVolume/shredos.x86_64/issues/21

Around 20% improvement in total throughput was seen with a block size of 128K (from 4k) on test hardware using an Atom D425. Further investigation found the CPU usage seems to be related to the OS's disk page cache behavior.

Running 4 dds on 4 different drives (similar to following) - Total disk throughput/CPU usage measured using iostat -x 5 is similar to nwipe. This means the system is bottlenecking things, not a fault of the nwipe write thread.

dd if=/dev/zero of=/dev/sda bs=1M

Adding oflag=direct (use O_DIRECT, bypass the page cache) improves throughput the the limit of the drives - around 150mb/s from 60mb/s. Total throughput is almost tripled on the test hardware.

Conclusion: Implementing O_DIRECT support is likely to significantly improve throughput on slow systems and reduce CPU usage for all machines. If O_DIRECT isn't possible, increasing c->device_stat.st_blksize provides some benefit.

PartialVolume commented 3 years ago

Agreed, my preference would be to test a O_DIRECT version of nwipe.

Slightly off subject, but so I can give you an idea in regards to my own schedule and where changing I/O method might fit in.

What follows are the issues I'm currently working on over the next couple of months. In order of priority. I'm currently working on 1.

  1. Fix corruption of footer when terminal resized, this also includes making the footer responsive to the current column width. I.e the following extra commands need to be added CTRL -A = Select all, I = info. So when the terminal column width is less than the length of the help text, nwipe prints ........... Ctrl-C more > flashing the > symbol at 1Hz to indicate pressing the right arrow key or > symbol will show more text. If the terminal column size is greater than strlen help text then display it in its entirety. This will cope with a changing terminal size such as when nwipe is used in a desktop environment.

  2. HPA detection and correction. I've been wanting to add this for a long time.

  3. O_DIRECT implementation

  4. New window info, to the right of the main window. The info window can be toggled on/off with 'i' and will contain information based on context. Such as during drive selection will show more detail about selected drive etc. During wipe method selection would show details regarding highlighted method. Being in its own window the text would be scrollable.