CMRR-C2P / MB

Support for CMRR multi-band pulse sequences
http://www.cmrr.umn.edu/multiband/
MIT License
57 stars 20 forks source link

R017 prerelease install: Vida XA31 Windows OS crashed during reboot after certificate install #313

Closed jcdicarlo closed 1 year ago

jcdicarlo commented 1 year ago

Since we have quite a few grants/projects that have been using MB on a Skyra with VE that is being replaced with a Prisma, we decided to try CMRR_MB_EPI_XA31A_R017pre5 on our Vida with XA31 and SP-02 update, but didn't get far. After adding the CMRR C2P certificate to Device Guard, reboot caused the Windows blue screen of exit or troubleshoot and ultimately a full software re-install by our Siemens FSE.

mharms commented 1 year ago

Weird. I'll just mention that I recently created a protocol on an XA31 Vida without any issues.

jcdicarlo commented 1 year ago

Weird. I'll just mention that I recently created a protocol on an XA31 Vida without any issues.

Did you need to add the certificate for kiosk mode, Michael? We had not had a version of this on our scanner, so I'm wondering if you had to do this per the Numaris instructions or were able to skip this step.

mharms commented 1 year ago

I didn't do the certificate install myself. The individual who did just told me that he did "the DeviceGuard in the Power Shell, followed by a reboot". It sounded pretty uneventful. Sorry, I can't be much help beyond that. My point was just to add a data point that things went pretty smoothly for us.

jcdicarlo commented 1 year ago

Happily, we had Siemens make an extra system backup yestereday and tried again with the 27-MAR build for XA31 CMRR_MB_EPI_XA31A_R017pre6, and were able to install without issue this time. (and without knowing why the certificate caused the issue last time, and if it was just a coincidence that it was the CMRR C2P being installed..)

neurosurd commented 1 year ago

I'm having a similar issue after installing the certificate for pre release 6 on XA30 SP02. It's taking me to the screen below upon reboot. I tried continue and turn off but it boots back to this screen. Any suggestions apart from calling Siemens for a complete reinstallation ? image

jcdicarlo commented 1 year ago

I'm so sorry- this is what happened initially to us and we called Siemens- our engineer determined the only way forward was a complete re-install.

neurosurd commented 1 year ago

Very strange. I've installed several certificates before and have not had issues.

eauerbach commented 1 year ago

It is really strange. Can you make sure your service engineer files a complaint and follows up on it, hopefully it can be escalated to someone who can figure out what is going on. I can't see how it has anything to do with the certificate itself; I have only ever distributed one certificate, it has never changed, there is no explanation for why it would suddenly start causing problems.

Note that you do not need to reinstall the certificate when updating the sequences. The certificate only needs to be installed once per system. Also note that I use the same certificate for all of the "CMRR" C2P sequences (MB-EPI, spectroscopy, FASTMAP, etc.), so it only needs to be installed once and it is good for all of them. The only reason ever to reinstall the certificate on a given system would be if the host has been replaced or had a full software reload. You will know if the certificate is missing if the installer .exe refuses to run (it is signed with the certificate).

Maybe attempting to install the same certificate twice is problematic? Seems like it shouldn't be, but I don't know if there is anything else common between the failures. There is a note in the certificate install instructions that the last step before rebooting can take 10 minutes or more to complete—which is of course ridiculous and unexpected—in these failure cases are we certain that the system wasn't rebooted prematurely?

neurosurd commented 1 year ago

Update: Siemens engineer had to reinstall the software. I was able to then successfully install the MB-EPI pre6 and Spectroscopy package using the certificate from the Spectro package, although this probably doesn't matter. Our engineer has posted a complaint within their system regarding this issue.

karl-koschutnig commented 1 year ago

I want to share our experience with installing the MB-EPI pre-6 release. We have a Vida (XA31) and followed the installation instructions closely step-by-step. Everything went great until we got the lovely blue windows screen after reboot (see screenshot above). Fortunately, we had a "fresh" Windows restore point, so we didn't have to reinstall the system completely. I encourage everyone to make a restore point BEFORE installing the new release. However, it seems that (at least in our case) the post of @eauerbach was the solution. Because we had already installed a prior release of the MB-EPI on our system, we completely skipped the certificate installation and directly installed the sequence itself. Everything work like a charm! This behaviour suggests that the second (or multiple) installations of the (same) certificate caused the problem. So maybe the developers could include a check whether this certificate is already installed or not. Or at least document this problem at the beginning of the install guide.

Good luck ;-)

jcdicarlo commented 1 year ago

I want to share our experience with installing the MB-EPI pre-6 release. We have a Vida (XA31) and followed the installation instructions closely step-by-step. Everything went great until we got the lovely blue windows screen after reboot (see screenshot above). Fortunately, we had a "fresh" Windows restore point, so we didn't have to reinstall the system completely. I encourage everyone to make a restore point BEFORE installing the new release. However, it seems that (at least in our case) the post of @eauerbach was the solution. Because we had already installed a prior release of the MB-EPI on our system, we completely skipped the certificate installation and directly installed the sequence itself. Everything work like a charm! This behaviour suggests that the second (or multiple) installations of the (same) certificate caused the problem. So maybe the developers could include a check whether this certificate is already installed or not. Or at least document this problem at the beginning of the install guide.

Good luck ;-)

Actually, when this happened at our site, we had no prior certificates on this system since it had only used stock sequences with the current software version; in our case it was not possible for multiple installations to cause the issue.

ken-sakaie commented 1 year ago

I want to share our experience with installing the MB-EPI pre-6 release. We have a Vida (XA31) and followed the installation instructions closely step-by-step. Everything went great until we got the lovely blue windows screen after reboot (see screenshot above). Fortunately, we had a "fresh" Windows restore point, so we didn't have to reinstall the system completely. I encourage everyone to make a restore point BEFORE installing the new release. However, it seems that (at least in our case) the post of @eauerbach was the solution. Because we had already installed a prior release of the MB-EPI on our system, we completely skipped the certificate installation and directly installed the sequence itself. Everything work like a charm! This behaviour suggests that the second (or multiple) installations of the (same) certificate caused the problem. So maybe the developers could include a check whether this certificate is already installed or not. Or at least document this problem at the beginning of the install guide.

Good luck ;-)

@karl-koschutnig

When you mention "windows restore point," are you referring to something like this?

https://support.microsoft.com/en-us/windows/create-a-system-restore-point-77e02e2a-3298-c869-9974-ef5658ea3be9

or are you referring to a Siemens system backup?

I just want to be prepared to protect our system from similar issues.

By the way, our Siemens contact concurs that re-installing the certificate is known to cause a blue screen and that it's our (the users') responsibility to avoid this.

Thanks! Ken

eauerbach commented 1 year ago

By the way, our Siemens contact concurs that re-installing the certificate is known to cause a blue screen and that it's our (the users') responsibility to avoid this.

Who said this? I have heard nothing but 🤷 from Siemens.

ken-sakaie commented 1 year ago

Our research manager. I will ask him to get the info to you.

karl-koschutnig commented 1 year ago

@ken-sakaie : yes, just a regular "windows restore point".

jmatthews-rotman commented 1 year ago

Our Siemens Research Collaboration Manager came on site today to oversee our second attempt at installing pre6 after the first attempt blue screened our system and our SFE had to re-image. Everything went smoothly today, though we did everything the same (already had pre5 on the system and followed the pre6 instruction exactly, including certificate install). He provided me with the exact same Windows Restore Point link that @ken-sakaie linked above, with the advice to set a restore point before any C2P install, and that the restore option from the blue screen should correctly restore the system without a Siemens FE needing to re-image. It seems worth adding a note to the C2P instructions that the certificate doesn't need updating between versions, at the very least to reduce the potential number of people running into this issue. It seems to be related to XA SP02 more than anything else. But not solely due to installing the same certificate multiple times, since I installed the cmrr mb pre5 and the cmrr spec separately (after our re-image), and then the cmrr mb pre6, which I now understand all share the same certificate, without issue. Maybe every system is destined to crash once after updating to SP02...

whitecraine commented 1 year ago

Hi, I have a Vida that is upgraded to XA50. Does anyone know if the same risk of crashing applies to XA50 as well?

mharms commented 1 year ago

I was talking with the individual that helped me with installs on both XA30 and XA31 systems (without any problems) about this thread, and one thought he had was whether the instances of problems might be related to attempting to install the certificate directly from a USB drive, rather than first copying the certificate onto the C: drive of the host itself.

neurosurd commented 1 year ago

Hi Michael, I can confirm that this was not the case at least at our site. All the installs were done from within C:/TEMP including certificate for other CMRR packages like spectroscopy which installed without issues.

m00ni19 commented 1 year ago

Hello we had exactly the same problem (blue screen/recovery mode) after trying to install the certificate. We have a Prisma XA30 and we had no problems during the installation last year. But this year the installation of the new license caused our windows to be corrupted.

We tried many ways to install it with the support of Siemens technicians and finally we succeeded!

Our hypothesis is that this problem could be related to the SD04 updates : When we tried the full installation of windows including the updates BEFORE the installation of the certificate it systematically failed!

So we suggest to : 1) Install windows without SD04 updates.
2) Install the certificate and license. 3) Install the SD04 updates.

Important : remember to make a backup of your data just in case and make recovery points to not have to reinstall everything from the beginning if needed.

eauerbach commented 1 year ago

Ok, Siemens has more or less confirmed that this issue is caused by the SD03 or SD04 update and hopefully it will be fixed with a new update in the next month or so. In the meantime I have updated the installation instructions included in the C2P archives with a quick test to determine if the certificate is already installed and an admonition to save a restore point before starting a new installation.

TakujiKinjo commented 1 year ago

Hi. I'm Takuji from Tamagawa University (we have Prismafit) in Japan.

We had faced some troubles concerning CMRR R017 pre5 and Siemens software ver. XA30 and SP02. When we tried physio recording, reconstruction had delayed and the system had crashed. We found, on this thread, such troubles had occurred on some other sites using Siemens XA30/31 software. It seems that the combination of pre5 and XA30 is not good at least for physio recording. Also, some of such sites seem to have replaced pre6 instead pre5 and succeeded, but some have failed.

Thus, we also replaced pre5 to pre6 before updating Siemens Security Delivery update to SD03, SD04. However, we still face the reconstruction problem I mentioned above with physio recording. Also, without physio recording the delay of reconstruction is more than that of pre5. Does anyone have any ideas for these reconstruction problems or know of any site which doesn't have any problems using the combination XA30 and pre6?

brainnetworklab commented 1 year ago

TakujiKinjo - have you had any success getting the sequence to run without the reconstruction problem?

@eauerbach - We're also running XA30 on a Prismafit. We're running pre6 resting-state with physio logging. We've tried all of the physio options (DICOM, file, legacy) and no luck. We initially tried a 64-channel head coil with the recommended settings to decrease reconstruction time (matrix optimization, View&Go Windows all closed). We tried the same options again on a 32-channel head coil and still have the reconstruction problem. We're certain it's related to physio because when we turn it off, the sequence runs just fine. We're running multiple resting-state runs and eventually it crashes the system. Any further suggestions?

TakujiKinjo commented 1 year ago

@brainnetworklab - No, not yet. The same problems as yours remain.

eauerbach commented 1 year ago

this issue is for the certificate related crashes, for the dicom physio see #306

eauerbach commented 1 year ago

Siemens has released SD05 for XA30 and XA31 which is supposed to fix this issue. I will update the installation instructions to reflect that you should not install a certificate on XA30 or XA31 unless SD05 is installed. Apparently XA20, 51, 51, etc. are not affected.

Check to see if a system is affected: The SDs installed can be found by clicking the gear icon on the top right of the scanner interface and selecting the Administration Portal. Under the Installation tab, select Software Update and change the Action Filter to “all”.

These are the fixed SD05 packages; if you see one of these you are safe to install:

VA30A-SD05:  MR035-23-R_0013162039
VA31A-SD05:  MR036-23-R_0013162152

For reference, these are the bad SD03/SD04 packages; if you see only these you should not attempt to install. Please contact Siemens service to arrange for SD05 to be installed:

VA30A-SD03:  MR061-22-R_0025270033
VA30A-SD04:  MR011-23-R_0001413951
VA31A-SD03:  MR060-22-R_0024553767
VA31A-SD04:  MR010-23-R_0001414062