IDER exceedingly slow?

[ndroftheline]

Hey Ylian and community,

Thank you so much for MeshCommander/MeshCentral.

I have an HP Elitedesk 800 G1, seems to work fine in general. I've updated its BIOS and ME firmware to the latest versions I could get my hands on (ME v9.1.42). MeshCommander from my Win10 box works just fine.

However I've had some struggles with IDER, my perception being that at certain points during, say, a Linux install process from an IDER-mounted ISO, speed drops to what apperas to be about 10-20 bytes per second, based on the status ribbon at the top of the meshcommander session. "IDE-R Session, Connected, 0 in, out."

As an example, right now I'm trying to boot a systemrescuecd iso and it's been booting for about 20 minutes, still going. I just did the same thing, from the same ISO from a usb 2.0 drive, and it booted in about 3 minutes.

I've attached the AMT Event Log, in case that's helpful at all. Is there any other troubleshooting/log investigation I can do to narrow down the problem? Or is this a problem at all?

IntelAmtEventlog--2020-02-18-07-41.json.txt

[Ylianst]

A few things on this from my past experience:

• IDER will send each disk request to MeshCommander and so, IDER is very sensitive to network latency. If you are over the Internet using CIRA, it's going to be really slow. I have seen WinPE take hours to boot, etc.
• Linux generally boots by creating a RAN disk, copy the kernel into it and launching the kernel. So, it loads lots of data at first and then traffic drops only read as needed.
• There are two IDER libraries that I use. IMRSDK.dll is the official Intel one and is used in MeshCommander .MSI version on Windows by default. I also have a fully JavaScript IDER library I built myself, that is the one used in MeshCentral or non-Windows MeshCommander. It's possible you will see different behavior from these. I would suggest trying both. In MeshCommander, IMRSDK is default, but if you hold SHIFT and hit the IDER button, it will switch to the JavaScript version.

I have had reports that the JavaScript version does not always work, but I have to spend time fixing it some more. In any case, let me know what IDER library you are using.

[MailYouLater]

As a workaround until IDER performance is improved, perhaps you could use IDER to boot a smaller system that can download and boot the desired ISO. Check out https://netboot.xyz to see if it would be suitable for your needs, the ISO is under 2 MB for the UEFI version and less than 1 MB for the non-UEFI version.

[jsastriawan]

Alternatively you can create a small IPXE ISO which points to an HTTP url containing installer ISO. I created this two hops solution for some clients OS recovery scenario. It works like a charm.

[Ylianst]

Wow, I did not know about https://netboot.xyz. Very interesting.

[jsastriawan]

Yeah.. this is definitely IPXE Iso with complete menu. This will definitely be saved on my toolbox. 😀

[ndroftheline]

Warms my heart to see so much knowledge and willingness to teach and share in forums like this. Thank you guys. I have played with netboot.xyz in the past, it's kinda scary voodoo magic to me tbh, and one of my "near-term" (ha) projects is setting up a netboot server from scratch to try and understand how the heck it works, hah. Anyway...

This is on a gigabit LAN, with only a simple/dumb layer 2 switch between the Win 10 and AMT boxes. I also have some Z230s and a Z240 here to do comparative testing with.

You're absolutely right, that the "speed" of the thing is at least partly a red herring based on quite a lot of faffing around with different distros. As you describe, a lot of traffic happens at what appears to be a reasonable speed until the IDER mount goes into what appears some sort of "heartbeat" mode while the kernel tries to probe its environment, not pulling any data from the virtual cdrom. Using the default IDER button and enabling text mode and debug options at the grub boot option of Systemrescuecd 6.0.7 on the elitedesk 800 g1, I get these errors during that "heartbeat" period (which is indefinite with systemrescuecd, but eventually Ubuntu Budgie 19.10 boots - Budgie reports the sda device as the intel virtual floppy):

sd 1:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
sd 1:0:0:0: [sda] Sense not available.

That just spams the kernel boot messages constantly. Some other relevant-seeming messages:

: : Mounting '/dev/disk/by-label/SYSRCD607' to '/run/archiveiso/bootmnt'
Waiting 30 seconds for device /dev/disk/by-label/SYSRDC607' ...
...
ERROR: '/dev/disk/by-label/SYSRCD607' device did not show up after 30 seconds...
   Falling back to interactive prompt
   You can try to fix the problem manually, log out when you are finished
[rootfs # ]

Ah! So it is an interactive prompt! After that point, system is still constantly spamming the [sda] errors so the interactive prompt is basically unusable.

Using Shift+IDER has resulted in some interesting behaviour, first the IDER session just seemed to stop on its own. I restarted the shift+ider session with the same ISO. Got to Systemrescuecd 6.0.7's GRUB menu. Began editing the default GRUB entry to enable text logging and debug mode, and in that short time the IDER session disconnected again (the green ribbon at the top showing info about the connection went away) and it wouldn't boot. Reconnected again, hit F10 and it might have started trying to boot...but then the whole AMT session dropped and now I actually can't connect to the machine via either web or meshcommander, nor ping it. Just seems the whole network system card isn't responding now. Powered it off, no change. Disconnected power and held power button to drain caps, waited a count of 15, reconnected power. Still unable to connect or even ping vPro/AMT subsystem. Powered on system by pushing button, and now can reconnect.

Attempted Shift+IDER mount of Systemrescuecd 6.0.7 again, noticed the default start option is "on reboot" instead of Graceful, switched it to Graceful, reset system, got to GRUB screen, set text and debug options, now getting similar error messages, but oddly they're scrolling much faster! Would you expect the experimental Javascript to be much faster? Took a picture to see some differences, now also seeing:

sd 1:0:0:0: [sda] Write Protect is on
sd 1:0:0:0: [sda] Mode Sense: 3e a3 b9 29
sd 1:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
sd 1:0:0:0: [sda] Sense not available.
sd 1:0:0:0: [sda] Write Protect is off

I'd like to file a bug with SystemrescueCD team, anything you can recommend I include there aside, obviously, from logs? Would you expect this behaviour from basically any AMT system or is it peculiar to these models/versions of ME?

let me know what IDER library you are using.

This is with the MSI-installed Windows MeshCommander app, v0.8.0.

Thanks heaps!

[billydu1980]

In fact, the most practical value of AMT technology is to be able to remotely repair/install the OS. Looking forward to install the Linux/Windows through IDER-mounted ISO.