Closed geerlingguy closed 12 months ago
Redirect added, and pages are up here:
Do you know if any RPi5 case that can be used with the HatDrive! Bottom? Pineberry has a picture on their website but it shows the HatDrive! Top variant I think.
I can't seem to find exact board outline dimensions for the bottom board. I was wondering if it would fit my enclosure...Is that published somewhere? I order a set btw and can verify that once they arrive.
A few quite notes testing the Top Hat:
sudo rpi-eeprom-update -d -a
to upgrade to the latest bootloader and rebooted. Apparently older bootloaders could have issues booting off NVMe...pi@pihat:~ $ sudo rpi-eeprom-update
*** UPDATE AVAILABLE ***
BOOTLOADER: update available
CURRENT: Thu 28 Sep 10:24:57 UTC 2023 (1695896697)
LATEST: Mon 30 Oct 16:45:10 UTC 2023 (1698684310)
RELEASE: default (/lib/firmware/raspberrypi/bootloader-2712/default)
Use raspi-config to change the release.
pi@pihat:~ $ sudo rpi-eeprom-update -d -a
*** INSTALLING EEPROM UPDATES ***
BOOTLOADER: update available
CURRENT: Thu 28 Sep 10:24:57 UTC 2023 (1695896697)
LATEST: Mon 30 Oct 16:45:10 UTC 2023 (1698684310)
RELEASE: default (/lib/firmware/raspberrypi/bootloader-2712/default)
Use raspi-config to change the release.
CURRENT: Thu 28 Sep 10:24:57 UTC 2023 (1695896697)
UPDATE: Mon 30 Oct 16:45:10 UTC 2023 (1698684310)
BOOTFS: /boot/firmware
Using recovery.bin for EEPROM update
EEPROM updates pending. Please reboot to apply the update.
To cancel a pending update run "sudo rpi-eeprom-update -r".
pi@pihat:~ $ sudo reboot
Another note: I could not get the Pi to boot off NVMe if I used rpi-clone
. I was able to do that on an earlier revision of the Pi 5 bootloader with the Pi prototype board... but a LOT has changed since then. It would try to boot, then drop me into a BusyBox initramfs shell. Very strange.
It works best if you flash Pi OS directly to the NVMe SSD, using the Pi with a USB adapter, Pi Imager direct, or on another computer.
A few questions about the board that I have now that I've put in a few hours...
What is the pitch of the FPC cable connecting the PCIe signal? It looks like 0.5mm?
dtparam=i2c_vc=on
You should see the INA with i2cdetect:
mgapinski@coralpi:/dev $ i2cdetect 0
WARNING! This program can confuse your I2C bus, cause data loss and worse!
I will probe file /dev/i2c-0.
I will probe address range 0x08-0x77.
Continue? [Y/n] y
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: 40 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: 50 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
The official active cooler can be used with the TOP Hat ?
Doe's significatively impact the cooling of the fan ?
Yes, and it does not seem like it is having a huge impact. It did not throttle when building the kernel
@geerlingguy as for your disk benchmark mentioned on X.
The 777 MB/s do not represent the device's capabilities since a) hdparm
uses a rather small block size when benchmarking (it's a tool from last century originally meant to deal with spinning rust) and b) you already showed that ~900 MB/s are possible with Gen3 speeds on RPi 5.
So either testing methodology or SSD in question is to blame for these low numbers (most probably both) :)
As for the 'testing methodology'... PiBenchmarks' Storage.sh
relies on hdparm
(limited block size, in the past it was 128K, with more modern versions it looks like 1M) or dd
with just 4K blocksize. Quick test with a SATA SSD should show the problem:
hdparm -Tt --direct /dev/sda
/dev/sda:
Timing O_DIRECT cached reads: 754 MB in 2.00 seconds = 376.91 MB/sec
Timing O_DIRECT disk reads: 1318 MB in 3.00 seconds = 439.28 MB/sec
dd if=/dev/zero of=test bs=4k count=130k conv=fsync
133120+0 records in
133120+0 records out
545259520 bytes (545 MB, 520 MiB) copied, 1.63406 s, 334 MB/s
Storage.sh
tells us about 334 MB/s sequential write speeds and 439.28 MB/s sequential reads. While when testing with e.g. 16M blocksize we're at 490/520 MB/s in reality:
iozone -e -I -a -s 100M -r 1024k -r 16384k -i 0 -i 1
kB reclen write rewrite read reread
102400 1024 107457 436871 441420 439296
102400 16384 486231 493974 519500 521398
No idea how you measured your 'nearly 900 MB/sec' on the Pi 5... but if it was only with Storage.sh
I would revisit this :)
A few test results from Pineberry, of various SSDs:
Edit: Also noticed they have a ton more up on PiBenchmarks.com: https://pibenchmarks.com/user/pineberrypi/
No idea how you measured your 'nearly 900 MB/sec' on the Pi 5... but if it was only with Storage.sh I would revisit this :)
@ThomasKaiser - I only use PiBenchmarks.com script because it's a fun way of gamifying the testing—for my actual reporting/testing I use my own script: https://github.com/geerlingguy/pi-cluster/blob/master/benchmarks/disk-benchmark.sh
It runs fio at 1024k block size 4 GB test, then iozone 1024k and 4k random read and write tests. It's quick but effective for most cases. I am looking at expanding the benchmarks a little now that so many devices are reaching into "GB/sec" range, and trying to go for more gigabytes could help squash any additional caching that's happening and throwing off the results.
The official active cooler can be used with the TOP Hat ?
Yep, see the video I just posted! https://www.youtube.com/watch?v=EXWu4SUsaY8
It runs fio at 1024k block size 4 GB test
Well, just 1024k is IMO nothing that represents today's sequential access patterns any more. Even network protocols like AFP or SMB dynamically tune their internal blocksizes to more than this in a LAN environment for over a decade now. See my SSD example above: with 1M it's 430/440 MB/s write/read while it needs 16M to get closer to the real sequential transfer rates: 490/520 MB/s
And a write test 4GB in size will result with el cheapo SSDs in an amount of data written to fast cache and once the cache capacity is exceeded a drastic drop in write performance can be observed which – with a 4GB test size – will result in a MB/s number neither representing the 'cache write performance' nor the 'sustained write performance once cache is full' but a crude mixture.
Firing up fio
with just 1GB data size but 4 times in a row generates both numbers and also an idea of how large the cache area of a specific SSD approximately is. Needless to say that once the cache area is exceeded the later iozone
tests will also suffer from this and generate almost meaningless numbers (at least 4k random I/O) :)
I want to try my FPGA load tester and see what the upper limit is. I can get about 25 Watts on the compute blade with the CM4. It was important to cool the CM4 to supply stable power otherwise the CM4 got to 80C very fast.
Does the pineberry hat resolve wifi issue with the prototype raspberry pi m.2 nvme hat?
Jeff used the old eeprom firmware with the OG prototype HAT. It had tons of issues
Does the bottom nvme board fit within the raspi case, with the cooler on the top?
Now you didn,t hear this from me, a birdie told me if you buy a spark fun qicc shim and bypass the voltage regulator you can i2c 5volts from pi to any accessories you want(pisugar, m.2[bottom] board, etc..)
Maybe a stupid question but is it mounted with the nvme to the rpi, or the nvme to the table? Can you make a picture?
@geerlingguy our docs with drive benchmarks are up on https://docs.pineberrypi.com/nvme-compatibility-list.
Some WD drives that did not work on the CM4 also fail on the Pi5, it is safe to assume most of them work.
Couldn't find a case, so I made a case (for the bottom hat). https://www.thingiverse.com/thing:6382978
@geerlingguy our docs with drive benchmarks are up on https://docs.pineberrypi.com/nvme-compatibility-list.
Some WD drives that did not work on the CM4 also fail on the Pi5, it is safe to assume most of them work.
Protip for anyone buying one of these hats - read that link! I didn't do that ahead of time, bought a Western Digital SN770M, and wasted a bunch of time and effort trying to make it work.
EDIT - Ordered a Sabrent and it works just fine with the HatDrive Top. Here's the specific model if anyone is interested. https://www.amazon.com/dp/B0BQG6JCRP
This drive scored 38493 using the PiBenchmarks script linked above with dtparam=pciex1 (gen 2) and 49078 with dtparam=pciex1_gen=3.
Where can I go to ask about hardware-issues related to the Pineberry Pi HatDrive! Top?
My Transcend 430s doesn't show up in lsblk
or fdisk -l
and I'm not sure, whether this is a hardware-incompatibility or just wrong configuration on my part (didn't have to deal with hardware-troubleshooting too much yet).
If this is a good place, I could post the journalctl
-output related to pcie. I also don't want to spam inappropriately.
EDIT:
Taking a closer look at the ssd-connector and the socket on the hat made me realize that I might have bought the wrong model of ssd.
I got both hatdrive modules for my raspberry pi 5 but sometimes I have the problem, that the ssd is not recognized on boot. I use a Western Digital SN530 wich is running perfectly if the hat recognizes it. To solve the problem I have to bend the ribbon cable and then mostly the ssd is reconized. I think the cables are too thin or defective. It occurs with both cables - the long one from hatdrive bottom and the shorter one from hatdrive top. Have you guys here had similar experiences?
Mine has been on since I got it, I've rebooted it multiple times. and I run Recalbox, but I upgraded the firmware and changed the boot order with the official OS first. I have no SD and just boot Recalbox from the M2. I didn't have an M2 to USB adapter so I booted from the SD and copied over the install image and ran the Raspberry Pi imager from the SD and installed the Recalbox image to the M2 that way. Has worked fine since.
I never tested the full Bottom kit until today—a few notes:
There are also some 3D printable case designs appearing on Printables now! https://www.printables.com/search/models?q=hatdrive
Just received my top and bottoms. It would be nice if the M.2 ssd could be screwed in from the top and not bottom, I hope that they will consider that in any future revisions. The supplied standoffs M2.5x6mm should be either M2.5x10mm or M2.5x6+6mm. This allows the bottom to then have extra stand offs on the bottom to keep the SSD off of the table/case or what ever you are using. Currently, for the bottom standoffs I am using M2.5x20+6mm nylon standoffs since the SSD has a heat sink.
for the bottom I have a Samsung 980 Pro + heatsink and for the top a micron 2400 2TB 2230.
The top came with a 28 pin gpio extender it would be nice if it included a 40 pin extender. Since I only have RPI 4's and RPI 5's I have no experience with the smaller gpios.
The installation went pretty smooth thanks to Jeff Geerling. Pineberry Pi did a wonderful job with the design of both hats.
Samsung 980 Pro Gen 3:
Category Test Result
HDParm Disk Read 794.56 MB/sec
DD Disk Write 371 MB/s
Score: 45184
tested 2 ssd with Hat Drive! Top on rpi5:
Union Memory not working, didn`t see nvme at all, size 2242
Kingston NV1, size 2280, is working, detectable, but size not correct
will buy different 2242 size ssd to test...
The HatDrive! Top works perfectly fine until I reboot the pi. Once I reboot or shutdown, the Nvme drive is no longer recognised until I unplug the pi from the power supply and boot it up again. This has happened with all 3 HatDrives that I have. Does anybody else have this issue? I am using WD SN530 Nvme drives.
The HatDrive! Top works perfectly fine until I reboot the pi. Once I reboot or shutdown, the Nvme drive is no longer recognised until I unplug the pi from the power supply and boot it up again. This has happened with all 3 HatDrives that I have. Does anybody else have this issue? I am using WD SN530 Nvme drives.
WD drives are tricky - this particular issue with the SN530 is listed on our docs page in the NVME compatibility section
I have similar problems (ssd not recognized at boot) with varoius drives. Unfortunately, my emails are ignored and my request to send new cables too... The drives work perfectly with the adapter from pimoroni.
I have similar problems (ssd not recognized at boot) with varoius drives.
Unfortunately, my emails are ignored and my request to send new cables too...
The drives work perfectly with the adapter from pimoroni.
It is not feasible to send spare ribbon cables knowing the costs of international shipping. We have sent our backlog now, so you'll get a reply soon - my inbox is flooded at the moment.
Hello Michal, I would have covered the costs, but of course it's fine this way - thank You very much!
The HatDrive! Top works perfectly fine until I reboot the pi. Once I reboot or shutdown, the Nvme drive is no longer recognised until I unplug the pi from the power supply and boot it up again. This has happened with all 3 HatDrives that I have. Does anybody else have this issue? I am using WD SN530 Nvme drives.
WD drives are tricky - this particular issue with the SN530 is listed on our docs page in the NVME compatibility section
Thanks @mikegapinski, I have been able to get it to boot from an SN530 on one of the HatDrives without any issues. Do you have an idea what the root cause of the boot issue is? If it is an issue that can be fixed in the rpi-eeprom, I can give it a shot and release it for other HatDrive users who maybe facing this issue.
The drive is not initialised in time and the bootloader doesn’t give. We don’t make the eeprom but there is one thing you can check - removing everything other than NVMe from the boot order. In theory it should reboot and retry.
RPI is constantly making changes to the bootloader so I expect those things to be ironed out in the future. We have decided to list the drives that are known to work to save people time on troubleshooting. We are also making some internal changes to the boards for future revisions but we don’t control the firmware on the pi itself
Wysyłane z aplikacji Outlook dla systemu iOShttps://aka.ms/o0ukef
Od: Shain Padmajan @.> Wysłane: Wednesday, January 17, 2024 9:48:37 AM Do: geerlingguy/raspberry-pi-pcie-devices @.> DW: Michał Gapiński @.>; Mention @.> Temat: Re: [geerlingguy/raspberry-pi-pcie-devices] Test Pineberry Pi's HatDrive! Top and Bottom NVMe HATs (Issue #559)
The HatDrive! Top works perfectly fine until I reboot the pi. Once I reboot or shutdown, the Nvme drive is no longer recognised until I unplug the pi from the power supply and boot it up again. This has happened with all 3 HatDrives that I have. Does anybody else have this issue? I am using WD SN530 Nvme drives.
WD drives are tricky - this particular issue with the SN530 is listed on our docs page in the NVME compatibility section
Thanks @mikegapinskihttps://github.com/mikegapinski, I have been able to get it to boot from an SN530 on one of the HatDrives without any issues. Do you have an idea what the root cause of the boot issue is? If it is an issue that can be fixed in the rpi-eepromhttps://github.com/raspberrypi/rpi-eeprom, I can give it a shot and release it for other HatDrive users who maybe facing this issue.
— Reply to this email directly, view it on GitHubhttps://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/559#issuecomment-1895350414, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHDCB7OQTNDANR6V4KSVQTYO6F6LAVCNFSM6AAAAAA7NGHADKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJVGM2TANBRGQ. You are receiving this because you were mentioned.Message ID: @.***>
bought sn740 today, size 2230, Pineberry HAT Top doesn`t detect at all
All WD drives are very similar to one another, we have 3 different SKUs that use the same PCB. That is why I decided to update our docs a while ago to say that we do not recommend ANY Western Digital drives at the moment. I have two SN530 drives in our lab and one of them only works. Different revision it looks like, consumer drives do not have a fixed BOM so they change controllers/nand from batch to batch
All WD drives are very similar to one another, we have 3 different SKUs that use the same PCB. That is why I decided to update our docs a while ago to say that we do not recommend ANY Western Digital drives at the moment. I have two SN530 drives in our lab and one of them only works. Different revision it looks like, consumer drives do not have a fixed BOM so they change controllers/nand from batch to batch
I think problem is in "pci gen4 x4", as i saw that sn520 wd drive working good with pineberry hat's, but it is "pci gen3 x2", already bought on ebay one of this 10, will test when will be received :)
Also maybe all drives with key M doesn`t work, because all pci gen4 x4 are without key b, only with key m, but older ones with pci gen3 x2 have b+m key....
I will test some Samsung and Kingspec SSDs next week and I will report the results of my tests.
Samsung devices are OK in general, but I've seen some complaints about Kingspec through our support channels. I don't know this brand at all, but we could test their lineup if they are popular
@geerlingguy
How is the 5V power input on the 'Bottom' version of the board to be used? Will some sort of adapter cable be included?
@mikegapinski
- We'll stock it but I have not found a device that needs it yet. We are playing it safe in case someone needs it, that is why we have power monitoring in the first place.
I can't get Samsung 990 PRO w/ Heatsink PCIe 4.0 NVMe SSD 4TB to work with HatDrive! Bottom. This may be the first drive that needs the additional power, where can I order this?
Hi Jeff & All!
I finally finished my Pi 5 setup with Pineberry Pi HatDrive! Bottom and Lexar NM620 256GB.
It took me a while because at first I got stuck with WD SN530. I couldn't force this little guy to boot but it seems this issue it's already mentioned on PineberryPi website. So I ordered Lexar NM620, tried again and it worked.
Thanks to Jeff's rpi-clone script I was able to transfer everything from microSD card to Lexar NVMe and finally boot from it.
I am pretty happy with this setup so far!
Here are some outputs using Storage.sh for LEXAR NM620 256GB.
PS. Greetings from Poland 🇵🇱
Score: 39051
Category Test Result
HDParm Disk Read 432.48 MB/sec
HDParm Cached Disk Read 347.03 MB/sec
DD Disk Write 277 MB/s
FIO 4k random read 108936 IOPS (435744 KB/s)
FIO 4k random write 76992 IOPS (307969 KB/s)
IOZone 4k read 135655 KB/s
IOZone 4k write 169752 KB/s
IOZone 4k random read 69211 KB/s
IOZone 4k random write 183361 KB/s
To copy the microSD with Raspberry Pi OS You can use "SD Card Copier" under "Accessoires" in the start menu.
To copy the microSD with Raspberry Pi OS You can use "SD Card Copier" under "Accessoires" in the start menu.
Yes, but only using the GUI. I needed a shell script in Raspberry Pi OS Lite and rpi-clone
worked just fine.
Might I suggest taking down WD SN530 from compatibility list - or at least mark it with red color or something... I just bought Pi5 + Pineberry NVME HAT + org. power supply + one of the first recommended drives (SN530) from the list only to have it not boot (of course the fault is mine cause I didn't read bottom of the page). Might save some frustration for future buyers. Found some old SK Hynix that works fortunately.
Our docs say:
WD SN530 has been problematic for some users when it comes to NVMe boot, especially in the 2242 variant. Our test drive was a 2280 SKU and it worked OK but we generally recommend staying away from Western Digital drives completely.
Od: KombatPL @.> Data: piątek, 26 stycznia 2024 o 20:34 Do: geerlingguy/raspberry-pi-pcie-devices @.> DW: Michał Gapiński @.>, Mention @.> Temat: Re: [geerlingguy/raspberry-pi-pcie-devices] Test Pineberry Pi's HatDrive! Top and Bottom NVMe HATs (Issue #559)
Might I suggest taking down WD SN530 from compatibility listhttps://docs.pineberrypi.com/nvme-compatibility-list - or at least mark it with red color or something... I just bought Pi5 + Pineberry NVME HAT + org. power supply + one of the first recommended drives (SN530) from the list only to have it not boot (of course the fault is mine cause I didn't read bottom of the page). Might save some frustration for future buyers. Found some old SK Hynix that works fortunately.
— Reply to this email directly, view it on GitHubhttps://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/559#issuecomment-1912588000, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHDCBYSVABRDN7DVHTJHGLYQQAK3AVCNFSM6AAAAAA7NGHADKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJSGU4DQMBQGA. You are receiving this because you were mentioned.Message ID: @.***>
I know I just got confused because this was one of the first positions on the compatibility list :) Like I said - fault is mine - I didn't read till the end before placing order (only read it after troubleshooting it for 2 days lol), but It could save someone else having to walk that path if it Was at least marked 'READ BELOW' :)
Pineberry Pi is a joint venture between Michał Gapiński and Mirosław Folejewski — both of whom have a history of making some pretty cool Pi products.
They just announced their "HatDrive!" Line of HATs (one for the top, one for the bottom) for the Raspberry Pi 5.
I have one of each and will be testing it, along with their impedance-controlled FFCs (they make two lengths):
I should also redirect the page https://pipci.jeffgeerling.com/hats/mirko-hat5m1-hat.html to these new HAT pages. See #550