Update 'External USB drives' page, rename to 'benchmarks'

geerlingguy commented 4 years ago

I have been doing a number of USB benchmarks recently, and finally with the Pi 4, the included 2x USB 3.0 ports offer a massive improvement—in many cases—over using microSD storage. In the past the USB 2.0 ports on the Pi 3, 3+, 2, etc. were severely hampered by no UASP support, 480 Mbps top bandwidth, and sharing a bus with the network jack.

See related issues with some of the data I've compiled so far:

https://github.com/geerlingguy/turing-pi-cluster/issues/11
See 'Pi 4 USB Boot SSD' numbers file on my local computer
See 'Pi 4 and CM3 64-bit Benchmarking' (for Pi 3+ comparisons) numbers file on my local computer

I'd like to update the External USB drives page with this new information (over in the https://github.com/geerlingguy/drupal-for-kubernetes repository), and rename the page to "External USB benchmarks"

So here's the checklist for this issue:

[x] Finalize the benchmarking script for each of these devices (copy this script to usb-benchmarks.sh, maybe? And allow passing a device name or something...)
[x] Benchmark all of the contenders in the table below.
[ ] Update the pidramble wiki USB drives page with the latest numbers
[ ] Update the page title to "External USB benchmarks" (and add redirect / new path?)
[ ] Publish the changes to www.pidramble.com
[ ] Compile the results into a slick video for YouTube :-)

So I'd like a comprehensive overview of USB 3.0 performance on the Pi 4 with a variety of devices.

Contenders

Product	USB 3.0 Support?	UASP (on Pi 4)?	Price (Total)
TDBT m.2 NVMe Enclosure + XPG SX6000 Lite 128 GB	Yes	Yes	$64.64
Inateck SATA enclosure + Kinsgston 240 GB SSD	Yes	Yes	$48.98
Corsair GTX 128 GB USB 3.1 flash drive	Yes	Yes	$55.99
Arcanite 128 GB USB 3.1 flash drive	Yes	No	$23.99
SanDisk Ultra Fit 128 GB flash drive	Yes	No	$21.99
SanDisk Ultra Flair 16 GB flash drive	Yes	No	$6.69
Samsung 32GB Evo+ microSD	No	No	$9.10

geerlingguy commented 4 years ago

Benchmarks

On the Pi, I'm running the usb-benchmarks.sh script, and using the External USB Drives information to format and mount the drives.

On the Mac, to overcome caching limitations, I used the following commands:

dd if=/dev/zero of=/Volumes/$VOLUME_NAME/test bs=8k count=50k conv=sync; rm -f /Volumes/$VOLUME_NAME/test
./iozone -e -I -a -s 5000M -r 4k -i 0 -i 1 -i 2 -f /Volumes/$VOLUME_NAME/iozone

The devices were formatted as "Mac OS Extended (Journaled)" with the "Master Boot Record" scheme.

Network File Copy

For the network file copy benchmark, I am using this 10 GB file. I have it on my Mac in my Downloads folder, and from the Mac, I run the command:

time scp ~/Downloads/10GB.bin pi@PI_HOST:/$VOLUME_MOUNT_NAME/10GB.bin

geerlingguy commented 4 years ago

Results

On Raspberry Pi 4, USB 3.0:

Product	`hdparm`	`dd`	4K read	4K write	Net Copy
TDBT + XPG NVMe	337.59 MB/s	217.33 MB/s	22.12 MB/s	34.23 MB/s	209.38 s (47.8 MB/s)
Inateck + Kingston SSD	299.65 MB/s	222.33 MB/s	16.35 MB/s	17.53 MB/s	247.38 s (40.5 MB/s)
Corsair Flash Drive	343.85 MB/s	216.67 MB/s	17.94 MB/s	18.03 MB/s	208.38 s (48.1 MB/s)
Arcanite Flash Drive	323.32 MB/s	176.00 MB/s	9.68 MB/s	3.72 MB/s	221.49 s (45.2 MB/s)
Sandisk Ultra Fit¹	67.38 MB/s	15.3 MB/s	4.99 MB/s	0.93 MB/s	633.21 s (15.8 MB/s)
Sandisk Ultra Flair	96.31 MB/s	13.97 MB/s	6.21 MB/s	0.69 MB/s	784.44 s (12.8 MB/s)
Samsung Evo+ microSD	48.83 MB/s	34.70 MB/s	10.43 MB/s	3.72 MB/s	483.79 s (20.7 MB/s)

On MacBook Pro, ThunderBolt 3:

Product	`hdparm`²	`dd`	4K read³	4K write
TDBT + XPG NVMe	N/A	351.82 MB/s	N/A	232.08 MB/s
Inateck + Kingston SSD	N/A	301.95 MB/s	N/A	166.15 MB/s
Corsair Flash Drive	N/A	286.09 MB/s	N/A	209.02 MB/s
Arcanite Flash Drive	N/A	255.87 MB/s	N/A	58.58 MB/s
Sandisk Ultra Fit	N/A	87.10 MB/s	N/A	5.94 MB/s¹
Sandisk Ultra Flair	N/A	18.16 MB/s	N/A	13.80 MB/s
Samsung Evo+ microSD	N/A	26.00 MB/s	N/A	24.24 MB/s

Also for fun, tested on internal SSD:

Product	`hdparm`²	`dd`	4K read³	4K write
Internal Mac SSD	N/A	589.84 MB/s	N/A	338.43 MB/s

¹ Note: The Ultra Fit got extremely hot, and seemed to have radically inconsistent test runs. It seems that the little device was built to optimize for size constraints, and overheats very easily. To get more consistency, I had to wait about a minute between tests. Under the 10 GB network file copy it went down to ~2-3 MB/s for much of the copy.

² hdparm is not available on macOS.

³ I couldn't get the 4K read to have reasonable results, so I must've been hitting some caching and even with a 5 GB file size, it wouldn't show more normal results for the reads.

geerlingguy commented 4 years ago

A quick note comparing the worst versus the best in terms of consistency—a lot of people might not think as much about this, but the thermals are incredibly important. When parts get way too hot and they throttle, you leave a lot of performance on the table.

Here's the ultra fit doing a stress run:

ultra-fit-overheated

And here's the Corsair GTX:

corsair-flash-even-heat

At least 4x better performance, and half the heat due to the case basically being a massive heat sink.

stale[bot] commented 4 years ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

Please read this blog post to see the reasons why I mark issues as stale.

stale[bot] commented 4 years ago

This issue is no longer marked for closure.

faceless2 commented 3 years ago

Jeff, thanks for the code. I don't know if you're collecting these, but if so: from 4 runs on Pi 4, discarded first, averaged:

Product	hdparm	dd	4K read	4K write
Kingston DataTraveler G4 32MB	148 MB/s	15.7 MB/s	9.3 MB/s	5.8 MB/s

geerlingguy / raspberry-pi-dramble