Closed sateuwdie closed 7 months ago
Thank you for your report!
The immediate power off behaviour is normal as the EC firmware needed an update as well, this cannot be prevented.
We asked our team to stop the updates from being public, as there are some critical issues with the latest versions.
Nevertheless, I believe it can be ignored safely, as this is probably a result of the entire BIOS region being flashed. I also believe that the blue screen you saw was a message that the firmware could not verify its integrity because of a newer EDK II version. This hypothesis hasn't been verified by the developers yet, though.
Please contact us directly in case you need instructions to downgrade to v1.5.0.
If you do not experience any critical issues with v1.7.0, you can safely continue to use it. A hotfix release (v1.7.1) will be published very soon.
@sateuwdie Thanks for the report.
Can you please clarify a few more things:
Thanks for answer. I reply to all
a) No critical issues
b)Yes, the blue screen image was this
and it show it only one time after firmware-upgrade.
c)the previous firmware version was 1.6.0 now report correct version 1.7.0
I got the same message when upgrading from v1.6.0 to v1.7.1.
@wessel-novacustom Did you have the same exact 0x5F error as on the screen above?
Some ideas from discussions so far:
@macpijan related issues
There are a number of issues regarding this feature, IMHO, which means there may be some design flaws. What bothers me is the fact that we easily display messages to scare users, but we do not provide enough guidance on the real meaning of the problem and recovery mechanics.
So, we implemented a protection mechanism with poor detection quality (many false positives) and no recovery. Every design of security features should treat all stages of protection, detection, and recovery equally seriously.
If, for some reason, we have a problem identifying why we have an issue, we at least should give some diagnostics tools or procedures to users so they know what they should do in such a situation.
I just reproduced this issue on the NS51PU-i7 after flashing v1.7.1
I believe if we did updates only for the RW_SECTION_A/B regions, the issue would no longer apply. Updating the RO partition will cause problems such as these. It also makes sense from the security perspective and would allow us to use the SPI chip's flash protection.
This is a big advantage that Vboot gives us and we're not making use of it.
@macpijan I think we should try that tomorrow.
I believe if we did updates only for the RW_SECTION_A/B regions, the issue would no longer apply.
We know that already, but this is not a solution to this particular problem.
Updating the RO partition will cause problems such as these.
Still, we need more in-depth understanding of this particular problem.
This is a big advantage that Vboot gives us and we're not making use of it.
We already take advantage of vboot, and try to minimize the necessity of updating RO partition. Here we had very good reasons to do so (like using Dasharo vboot keys), and you know in reality we may face others - like the necessity to extend the flash layout to fit 2 FSP copies, etc.
detection quality (many false positives) and no recovery
This is not really proven yet. Maybe we do in fact boot from recovery in these cases, so the message would be true. We need to have a closer look at the case when reproduction happens.
This is not really proven yet. Maybe we do in fact boot from recovery in these cases, so the message would be true. We need to have a closer look at the case when reproduction happens.
@macpijan Yes, the technical root cause is not yet proven, and I can't comment on the reason in the code because I don't know this implementation so well. My comment is purely how it seems from a UX perspective. We need better tests of this feature.
@macpijan upgrading from v1.7.0 to v.1.7.1 does not reproduce this issue. How to get DTS image which upgrades from v1.5.0 to v1.7.0? My guess is the DTS v1.2.10 but the files are nowhere to be found. I would like to use the same procedure when reproducing it, especially that the error occurred only once for unknown reason (the hash space lock was in v1.6.0 too).
@miczyg1 Wessel has been able to reproduce it going from 1.6.0 to 1.7.1 with the DTS available in iPXE. It does not happen always, though...
It does not matter if we are going to 1.7.0 or 1.7.1 from 1.6.0. In both cases the bios region gets flashed.
especially that the error occurred only once for unknown reason
This error has happened multiple times already, for Marek, Wessel, and various end users.
This error has happened multiple times already, for Marek, Wessel, and various end users.
I meant the error occurs only once after flashing.
So all it takes is to flash v1.6.0 back and update from DTS again
v1.6.0 -> v.1.7.1 still no reproduction.
Found possible culprit: https://github.com/Dasharo/coreboot/blob/dasharo/src/mainboard/clevo/adl-p/cmos.layout#L22 perfectly overlaps with https://github.com/Dasharo/coreboot/blob/dasharo/src/soc/intel/common/basecode/ramtop/ramtop.c#L35
Having such luck is beyond coincidence...
I will need to test this on a brand new device that has never had Dasharo installed before. For example, by installing v1.6.0 on NS5x 12th Gen, install the operating system and use it normally. Then I will install the test binary with the command flashrom -p internal --ifd -i bios -w /path/to/coreboot.rom
to see if I get the message again.
Another instance of this problem was reported on Dasharo - Support
channel. Fixing it is one thing, but correctly communicating such a scary message may pop up is another. IT would be great if we can address both, because this bug really damage Novacustom and Dasharo reputation.
Another instance of this problem was reported on Dasharo - Support channel
Yes, that was me. :)
I am a relatively naïve non-techie user who updated firmware for the very first time so allow me to give respective UX feedback.
What could be (or could have been) better:
I hope this is informative for you, I'm just trying to help since I wholeheartedly support what you all do! 👍
I noticed some other things I'd desire regarding the general documentation of a firmware update, but this would be offtopic here. Let me know if and where I should comment on that. :)
@sycam0r-e Thank you for your valuable feedback! I hope @macpijan can pick this up and improve a bit were needed/agreed.
I noticed some other things I'd desire regarding the general documentation of a firmware update, but this would be offtopic here. Let me know if and where I should comment on that. :)
Your insights are highly appreciated, @sycam0r-e, thank you for your contributions!
Let me know if and where I should comment on that. :)
Please consider creating issues on our Dasharo Issues page.
You can conveniently create new issues using this link: Create a New Issue. Providing detailed context and information, including screenshots or photos, will help us address and resolve the issues effectively.
Feel free to join our Matrix community: https://matrix.to/#/#dasharo-laptops:matrix.org to share your thoughts.
@mkopec Has this been verified? I can imagine it is hard to reproduce ...
Allow me to direct your attention to one of the more important parts of my earlier message that likely just got lost:
As far as I understand your current thinking is that the warning message being shown is erroneous and that users have actually nothing to worry about. However, this is not stated clearly in this issue. Make a separate comment in this issue detailing under which conditions (e.g. if the iPXE boot menu has been used) users do not have to worry.
Emphasis added.
@macpijan This issue hasn't been resolved yet. It was closed, but can be opened again.
The thing is that it only seems to happen once per device when upgrading from version < v1.7.0 to version >= v1.7.0. In our production, we always apply version v1.6.0 and then upgrade to version v1.7.2. In all cases, we still get this error, even though we use the latest version of Dasharo Tools Suite. Do you have any ideas how we can solve this issue? NovaCustom users report a lot that they are scared of the error message.
Issue has been reopened, as it seems the problem has not been solved, and it still appears in the following scenario:
In: https://github.com/Dasharo/dasharo-issues/issues/556#issuecomment-1809940586 there was a potential fix, but it seems that the problem still appears on the new hardware units.
@macpijan Exactly.
As we have new hardware all the time, we can do testing for this issue.
I believe I hit this issue on MSI Z690 too. Updated from 1.1.1 to 1.1.3 using flashrom -p internal -w [path] --fmap -i RW_SECTION_A -i RW_SECTION_B
, then typed "reboot" and got message like this:
After pressing ENTER, I immediately entered setup, and was greeted with 1.1.1 version there.
@marmarek I believe the command for updating to v1.1.3 should be flashrom -p internal --ifd -i bios
according to the DTS scripts: https://github.com/Dasharo/meta-dts/blob/017ec32aab3f4481c3a904d507212fcf73570cc7/meta-dts-distro/recipes-dts/dts/dts/dts-functions.sh#L323
The firmware update docs need to be updated to reflect this: https://docs.dasharo.com/unified/msi/firmware-update
hmm, maybe better describe it on https://docs.dasharo.com/unified/msi/firmware-update/ then? the section "Version v1.1.0 or newer" says to do just RWSECTION{A,B} (there is a note about 1.1.2 too, but it's unclear...)
Yes, the docs appear to be outdated, see my edit above. Need to update the docs so that the entire BIOS region is flashed.
ok, after updating entire BIOS region, correct version is running, but still got the message after update.
The error indicates that Vboot could not save the hash of recovery MRC cache to TPM NVRAM. Since this error appears once per machine lifetime (after switching from Insyde), it may mean that we need to do a reset of the TPM during initial deployment. For upcoming releases we may work around this issue by turning on secdata mocking.
@miczyg1 wdyt?
We've sent binaries for testing to @wessel-novacustom , please let us know when you're able to confirm that the issue is fixed now.
We've sent binaries for testing to @wessel-novacustom , please let us know when you're able to confirm that the issue is fixed now.
I'll do tomorrow at latest.
I have tested the binary on an NV41PZ running Dasharo v1.6.0 (never been to v1.7.x on that unit before) with the following DTS commands:
wget http://192.168.1.75/coreboot.rom
(this is from a local server of course)
sha256sum coreboot.rom
says: ec05de09b986b45fb0d1e77262777c95a5a4f7f1ea58952dcbab30c0daf09bb6 coreboot.rom
I have flashed the binary with the following command:
flashrom -p internal --ifd -i bios -w coreboot.rom && systemctl poweroff
After a few minutes, the laptop has shut down.
I powered it on and the message did not appear as it usually does after an update.
I have tried this on multiple devices. With both EC of version v1.6.0 and EC of version v1.7.2, with the same results.
I have even updated manually from v1.6.0 to the original v1.7.2 version on an NV41-ADL that again never had Dasharo v1.7.x before. I did this to make sure that the DTS migration process of the original update method isn't causing the issue. The pop up showed up at the first boot, which is not the case for the test binary.
So yes, we have the sweet spot, it seems.
Closing, as the issue is confirmed to be fixed. I also posted some info in a related issue: https://github.com/Dasharo/dasharo-issues/issues/689#issuecomment-1981496430
@wessel-novacustom @mkopec It looks like the issue is not fixed yet because today I updated the firmware from version 1.6.0 to 1.7.2 on my NV41 Series laptop and the blue warning appeared once.
@wessel-novacustom @mkopec It looks like the issue is not fixed yet because today I updated the firmware from version 1.6.0 to 1.7.2 on my NV41 Series laptop and the blue warning appeared once.
It has been fixed for new releases, but there hasn't been any new stable version released yet.
Maybe @mkopec or @macpijan can confirm.
Dasharo version 1.7.0
Dasharo variant Dasharo (coreboot+UEFI) v1.7.0
Question text Today I have upgraded the firmware via ipxe from dasharo tools suite, version: 1.7.0 The update apply but the system poweroff immedately during update (of course was connected to AC source) i press the power button and system come up..with a blue screen tell me firmware was not updated correctly. The system boot, all works, no error reported, I can enter in coreboot menu, etc. Can I stay safe? Thanks The system is NS5x_NS7xPU