Availability of firmware version P16

[davidspek]

Sorry for writing this in an issue, however, I am not able to create an STH account for some strange reason. I am running Unraid and have just ordered an H710 mini mono to replace my H310 full-size card. For a long time, TRIM has not worked on these cards within Unraid even though my SSD (1TB 860 EVO) meets the requirements from LSI for TRIM. The solution was solved by flashing the P16 firmware onto the card, after which TRIM has worked flawlessly.

Would changing the firmware version flashed by the tool be as simple as replacing the firmware file? Or would there be more steps involved in this process?

Thank you for your time and great work.

[Fohdeesha]

To avoid issues, I would follow the guide all the way through unmodified. After that though, it's an official LSI card, so once done (after setting the sas address), you can just use the official LSI tool to flash P16 over the latest version the guide flashes. You can even use the same live linux ISO, just wget in a p16 firmware file and run:

/root/sas2flash -o -f p16.bin

then reboot, it should be good to go.

[Fohdeesha]

also I would open a ticket with unraid if someone hasn't already, needing 10 year old firmware just for TRIM support seems like a major issue. the latest LSI FW has working TRIM in freenas, I know that much at least. I'm also not an unraid expert, but some cursory googling shows there were trim issues with SAS2008 based cards, while the H710 is SAS2308 - perhaps try with the latest firmware first and see if trim is shown as supported?

[Fohdeesha]

(your previous H310 was SAS2008 based, so it would have indeed been included in those trim issues)

[davidspek]

Thanks for looking it too all that and your recommendation. For some reason, I thought the H710 would also be affected. I looked into it for a long time so maybe there is some page somewhere that also mentioned the controller of the H710, or I'm just mixing them up. I will report back in this issue how things have worked out once the card comes in so others can also find it later on.

[Fohdeesha]

I could be totally wrong, I've never used unRAID before. But some of the google results I found specifically mentioned the SAS2008 chipset (which your H310 had), while the H710 is the much newer SAS2308. I also found the below, so who knows:

"Trim on LSI SAS2008/2308 hasn't been working for a while, it appears to be a kernel/driver issue, it works on the newer LSI SAS3008 based controllers,"

[davidspek]

I'm guessing that is the comment that I read. It is a long ongoing issue and I wonder why it hasn't been fixed.

[davidspek]

Would you have any input on this comment I found?

"Bear in mind for those with SAS2008 HBAs that want to downgrade to P16; its not a straightforward sas2flsh.exe -o -f 2118it.bin. These are the requirements I've found but the list is by no means exhaustive:

1. You will need to clear the SBR and the Flash using megarec first. Make sure you note your SAS address prior to doing so!
2. If running a PERC card (H200 in my case) you will need to flash a suitable Perc Firmware file THEN reboot THEN flash the P16 firmware file.
3. Make sure you have a suitable version of sas2flsh that supports downgrades as well as IR to IT changes if you require this. I've found version 5 to work for me but YMMV."

[Fohdeesha]

OK, if that's the case, you will need to modify the flash guide a bit indeed so it flashes P16 off the bat (as you can't just clear the SBR and boot back into linux and downgrade as that suggests, with the mini mono cards that will brick them - the server will not boot if a card in the mini slot does not have a valid dell SBR)

basically, follow the guide all the way up until the linux flashing part, the B0-H710 or D1-H710 command (which one the guide tells you to run depends on the revision of your card). So get up to that command, but don't run it yet. wget in a P16 file from somewhere. Those commands pull from /root/H710/9205-8e.bin and /root/H710/9207-8.bin - again, depending on which revision you have. Overwrite whichever one is applicable with your P16 version of it, then you can continue the guide, and the B0-H710 or D1-H710 command will now use your P16 image. You can see the scripts the live ISO uses here if it helps clear things up: https://github.com/Fohdeesha/lab-docu/tree/master/Perc%20Guide%20Source%20Files/Linux/usr/local/bin

[davidspek]

That's what I expected I would need to do. I've just downloaded a copy of the P16 firmware. Would there be interest in me merging a D1-H710-P16 script that points to a different directory? I assume it will just take duplicating the original one and changing the file path. I could merge that edit quite quickly (though I can't test it just yet).

[Fohdeesha]

We could, but I would need to rebuild the live ISO, create a new download package, push an update with the new MD5SUM to the guide, etc. I'd probably hold off on that until we see someone else needing to do this as currently you're the only one :)

[davidspek]

Alright, no problem. One more quick question, I see there is only 1 set of BIOS and UEFI roms in the directory. Would those need to be changed to older versions as well to your knowledge? The P16 package from broadcom is missing the UEFI rom so I was wondering if the one in the script might be incompatible with the P16 firmware.

[Fohdeesha]

I'm pretty sure you need to match the bootloader version with the firmware version, Broadcom's site is a fucking mess, it seems back then they packaged the EFI bootloaders separately, but here it is: https://docs.broadcom.com/docs/12348625

[Fohdeesha]

(and yes you'll need to copy the x64sas2.rom out of that zip and overwrite the one in the live ISO)

[davidspek]

So I am trying to do the flash, and ran through the entire process twice but something seems to not be going correctly. I do not have the SAS cable connected at the moment but I doubt that is what is making the difference.

[davidspek]

Connecting the backplane (with drives installed, and while no power was going to the server), then rerunning everything also gives the same error. During Linux booting I do see some errors with the driver loading it seems. Not sure if that is normal.

[davidspek]

These are the errors I see during the booting process.

[davidspek]

So I just tried with the normal firmware and that does seem to be working. I noticed a difference in the number of loaded bytes during the loading of the firmware and IOC Host Boot successful instead of IOC failed to become ready.

[davidspek]

Got everything flashed successfully and everything is working with P20 firmware. But as expected, TRIM is not working within unRAID.

Edit: Just to clarify, the SSD is formatted as BTRFS. This might be the reason why this issue isn't seen in FreeNAS.

[davidspek]

Does anybody have an idea why it would not flash with the P16 firmware? It seems like it wasn't wanting to boot the card from RAM.

[Fohdeesha]

It might be that P16 is too old to work with the ram booting feature of lsirec, other than that no clue

[davidspek]

I've been running the H710 with regular firmware since my last post and it has been working great. Today I finally got around to trying to get the P16 firmware loaded and it worked the first try.

What I did: I copied the 9207-8.bin P16 firmware to /root/H710/9207-8_p16.bin I edited line 24 of the script /usr/local/bin/D1-H710 from /root/lsiutil/lsiutil -p1 -f /root/H710/9207-8.bin -y 2 to /root/lsiutil/lsiutil -p1 -f /root/H710/9207-8_p16.bin -y 2.

This way, the card successfully booted from ram using the P20 firmware and then was flashed with the P16 firmware. I am assuming that this procedure would also work for a card that hasn't already been flashed with P20 before. I think giving others the option to run a D1-H710_P16 command would be a nice feature for everybody that runs BTRFS on their SSD's.

[beckzg]

Hi,

Is there a way/utility to list physical disk from Debian after the IT formware is successfull installed? Because the /dev/sdX is not populated in the same order then the physical slots are. Or at least I dont know the "logic" :)

I've an R620 with H710 Mini, I populated one - one SSD in the first and last slot and HDD to the remaining 6 slots. From Debian I see that sda and sde are the SSD-s. I tried to change the order of two cables, then the sdd and sde was the SSD.

I tried the megacli and megactl, but none of them works.

My feature problem is, that when a disk/ssd will fail, how can I figure out which one is the wrong?

Kind Regards, Zoltan

[davidspek]

Hi Zoltan,

Your question is not really related to this issue. But you can try using lsblk -f, then you will get the UUID of the disks. What OS are you running? You will still need some way to identify which serial number or UUID is in what slot.

[beckzg]

Sorry, my fault :)

I’ve Debian:

cat /etc/debian_version 10.3

My two SSD are in slot 0 and slot 7 in front of the server, so for example if the HDD in physical slot 3 will fail, then which one is from the list of Debian?

The IT firmware has no CLI?

lsblk -f NAME FSTYPE LABEL UUID FSAVAIL FSUSE% MOUNTPOINT sda
├─sda1
├─sda2 vfat 474B-7DF5
├─sda3 zfs_member rpool 8653612632864926491
├─sda4 zfs_member local-VM 10035377797163395202
└─sda5 zfs_member
sdb
├─sdb1 zfs_member local-VM 10035377797163395202
└─sdb9
sdc
├─sdc1 zfs_member local-VM 10035377797163395202
└─sdc9
sdd
├─sdd1 zfs_member local-VM 10035377797163395202
└─sdd9
sde
├─sde1
├─sde2 vfat 474C-22EC
├─sde3 zfs_member rpool 8653612632864926491
├─sde4 zfs_member local-VM 10035377797163395202
└─sde5 zfs_member
sdf
├─sdf1 zfs_member local-VM 10035377797163395202
└─sdf9
sdg
├─sdg1 zfs_member local-VM 10035377797163395202
└─sdg9
sdh
├─sdh1 zfs_member local-VM 10035377797163395202
└─sdh9

2020. ápr. 27. dátummal, 12:18 időpontban DavidSpek notifications@github.com írta:

Hi Zoltan,

Your question is not really related to this issue. But you can try using lsblk -f, then you will get the UUID of the disks. What OS are you running? You will still need some way to identify which serial number or UUID is in what slot.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Fohdeesha/lab-docu/issues/6#issuecomment-619883560, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD6PRMJQ3W4OS5WSVKWR6T3ROVLYHANCNFSM4LOW4CMQ.

[davidspek]

I think this link might be useful for you: https://serverfault.com/questions/64239/physically-identify-the-failed-hard-drive.

You can use smartctl to find the serial number and other info about a drive. That link also provides commands for turning on the identification lights on the drive trays.

[beckzg]

Thank you, I will give a try!

Kind Regards, Zoltan

2020. ápr. 27. dátummal, 12:30 időpontban DavidSpek notifications@github.com írta:

I think this link might be useful for you: https://serverfault.com/questions/64239/physically-identify-the-failed-hard-drive https://serverfault.com/questions/64239/physically-identify-the-failed-hard-drive.

You can use smartctl to find the serial number and other info about a drive. That link also provides commands for turning on the identification lights on the drive trays.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Fohdeesha/lab-docu/issues/6#issuecomment-619890044, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD6PRMIH2FDXNLGJSMBVZUTROVNEXANCNFSM4LOW4CMQ.

[Fohdeesha]

@DavidSpek hey, trying to close out old issues - can you refresh my memory, just adding a new duplicate script in the linux ISO, except lsiutil flashes p16 instead of p20 was enough to solve your issue? eg the script still live-boots the card with p20 firmware, but then writes p16 to it's flash? If so I'll see if I can get this added to the next ISO I build

[Fohdeesha]

I guess I should ask more importantly, does unraid still have this issue with no TRIM in p20 to necessitate this older firmware?

[davidspek]

@Fohdeesha Indeed, live-booting the card with P16 firmware did not work. So the card was live-booted with the P20 firmware, but the script then flashes the P16 firmware.

As far as I know, this is caused by a bug with BTRFS and the mpt2sas drivers (don't quote me on the driver name), and not isolated to unRAID. I don't believe there has been a fix, as this would have to happen upstream in the mpt2sas and I doubt people running into this error will be fixing it there. I think it would be nice to have an option to flash the P16 firmware in the next ISO and scripts, just for those that might need it. I don't think it would be too difficult to add. I could try to find some time to take a look at integrating it into the script, just let me know if you need the help.

[Fohdeesha]

Ah, got it. I think I'll just make a duplicate of both linux flashing scripts, titled something like B0-H710-P16 and D1-H710-P16, and change the ROM flashing line to use a P16 image. Then put a note in the guide noting that BTRFS users may want to use the P16 variant commands instead (and probably link to this git thread for background info)

[digitalspaceport]

Ah, got it. I think I'll just make a duplicate of both linux flashing scripts, titled something like B0-H710-P16 and D1-H710-P16, and change the ROM flashing line to use a P16 image. Then put a note in the guide noting that BTRFS users may want to use the P16 variant commands instead (and probably link to this git thread for background info)

Hello @Fohdeesha did the -P16 variant make it into the releases by chance? I am a BTRFS Unraid user also who really appreciates the quality easy flash experience you have created for H710's.

[Fohdeesha]

OK, I should have a new ISO in a couple weeks, but I've gone ahead and made a script that pulls in all the necessary files for a P16 flash, including new scripts that properly ram-boot the card then flash P16. Follow the online guide all the way up until it has you run the D1 or B0 flash commands in linux (you should be sudo'd to root). Then instead of running those to flash P20, just run:

curl -s https://fohdeesha.com/docs/store/perc/P16/P16-Fetch.sh | bash

That will grab everything and put it in the right place. Then just run either "B0-H710-P16" or "D1-H710-P16" depending on which card you have.The P16 bootloaders are also there if you need to boot off the card, instead of using the boot flashing commands in the guide (those will flash the P20 loaders) just run either:

#flash BIOS bootrom
flashboot /root/Bootloaders/p16-mptsas2.rom
#or flash the UEFI bootrom
flashboot /root/Bootloaders/p16-x64sas2.rom

that's it, everything else should be the same as what's already in the guide

[Fohdeesha]

OK, a new ISO was built and the guide site is now live with the new 1.9 ISO package. Just follow the guide site as normal, but when you get to the linux flash commands, just run ""B0-H710-P16" or "D1-H710-P16" instead of "B0-H710" or "D1-H710". That should be it! I'm going to leave this issue open until I get a moment to add an instruction note to the actual guide regarding P16 and how to flash it if you need it

[Fohdeesha]

Does anyone need P16 on the H310 variants? As of now the new ISO only has the components to get P16 on the H710s and H810s

[steeven-th]

Hello,

@Fohdeesha I can't flash the card. I have this problem. I tested with the P16 too. (Sorry for my bad English)

Thanks

[Fohdeesha]

yes - the new ISO is bad, for some reason lsiutil is not working on debian 11. follow the guide using this older iso package - https://fohdeesha.com/data/other/perc/perc-crossflash-v1.8.zip

it doesn't have the P16 files, but you can follow my earlier post above to grab the files once you've booted the ISO

[steeven-th]

I have follow your earlier post and I have this problem. I will try with ISO 1.8. Thanks

[steeven-th]

Everything seems to work with ISO 1.8 and the D1-H710 control

Thanks

[steeven-th]

@Fohdeesha is it normal that IDRAC doesn't see the physical disc after flash ?

[Fohdeesha]

yes, it's no longer a perc card

[steeven-th]

@Fohdeesha Hi, I have a problem. I can boot on USB key for install an OS (for me, it's Proxmox) but I can't boot on HDD after install. I have Try in BIOS and UEFI Do you have an idea ? Thx

[mentalinc]

Hi @Fohdeesha

Edit:just found the sth thread re non dell hardware, will test bios settings and new 1.9 iso and report back later.

Thanks for the great toolset. I'm trying to flash a B0-H710, I've run the clean successfully, but I'm not able to flash the LSI firmware with 'B0-H710' I get "IOC failed to become ready"

and then 0 MPT Ports Found, leading to not being able to 'setsas'

My live USB label is "Debian stretch 20200209-02:51"

I'm also getting the errors like posted above when in the LIVE ISO.

https://github.com/Fohdeesha/lab-docu/issues/6#issuecomment-603265087

Finally, windows device manager can see the LSI SAS 2308 Mustang but I get the follow error (I assume the flash has failed) "An I/O adapter hardware error has occurred."

Any ideas appreciated.

[mentalinc]

And to reply to my post above.; Solved the issue. IOMMU needs to be disabled in the AMD x570 bios Advanced - AMD CBS

Thanks to the page below where it notes in in relation to AMD-Vi errors I was getting. https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#Prerequisites

[Fohdeesha]

@mentalinc this is covered in the last couple pages of the STH topic, on the new ISO (and really, even the old one), VT-D, virt extensions, and anything else that might create IOMMU groups needs to be disabled (AMD-V / VT-D are just encompassing trademarks for their virtualization IOMMU implementations, disabling IOMMU alone may work on some platforms, but really it's more reliable to disable virt entirely during the flashing process): https://forums.servethehome.com/index.php?threads/guide-flashing-h310-h710-h810-mini-full-size-to-it-mode.27459/page-13#post-310361

[Kennochas]

Does anyone need P16 on the H310 variants? As of now the new ISO only has the components to get P16 on the H710s and H810s

I am stumbling onto this mess with a full-size H310 that I have IT mode flashed already, but Trim doesn't work. Currently trying to understand what I would need to do to get down.

[Fohdeesha]

I've generated a new ISO set with P16 scripts for both the H710s and H310s, it's live on the main guide. Just follow the guide on the website as normal, except when it gets to the instructions inside linux saying to run either "B0-H710", "D1-H710", or "H310", just append "-P16" to the command.

So for instance if you have an H310 Mini you'd run:

H310-Mini-P16

If you have a B0 H710, you'd run:

B0-H710-P16

If you have a D1 H710, you'd run:

D1-H710-P16

That's it, follow the rest of the guide instructions as normal. Except for the bootloader flashing commands, if you need to flash a bootloader, it needs to be the P16 version. That's included on the ISO, you would just run one of the below to suit your requirements:

#flash BIOS bootrom
flashboot /root/Bootloaders/p16-mptsas2.rom
#or flash the UEFI bootrom
flashboot /root/Bootloaders/p16-x64sas2.rom

That should be all you guys need, I'm going to go ahead and close this issue as solved but feel free to reply if something doesn't work

[Kennochas]

I've generated a new ISO set with P16 scripts for both the H710s and H310s, it's live on the main guide. Just follow the guide on the website as normal, except when it gets to the instructions inside linux saying to run either "B0-H710", "D1-H710", or "H310", just append "-P16" to the command.

So for instance if you have an H310 you'd run:

H310-P16

If you have a B0 H710, you'd run:

B0-H710-P16

If you have a D1 H710, you'd run:

D1-H710-P16

That's it, follow the rest of the guide instructions as normal. Except for the bootloader flashing commands, if you need to flash a bootloader, it needs to be the P16 version. That's included on the ISO, you would just run one of the below to suit your requirements:

#flash BIOS bootrom
flashboot /root/Bootloaders/p16-mptsas2.rom
#or flash the UEFI bootrom
flashboot /root/Bootloaders/p16-x64sas2.rom

That should be all you guys need, I'm going to go ahead and close this issue as solved but feel free to reply if something doesn't work

Just wanted to reply saying thanks, this definitely installed H310-P16. However, it doesn't appear to fix the trim issues, so that is kind of a bummer. Time to controller shop.

[igyacine]

I have follow your earlier post and I have this problem. I will try with ISO 1.8. Thanks

plz share this iso

[igyacine]

Hello, @Fohdeesha I folowed your instruction but still have this error. i've used perc-crossflash-v2, and D1 H710 0

[RarogCmex]

@Kennochas @Fohdeesha Confirm, there is no trim support for non-determenistic trim ssds even with 16.0 I hope TRIM command is not so important if you have plain tasks.