utmapp / UTM

Virtual machines for iOS and macOS
https://getutm.app
Apache License 2.0
26.85k stars 1.34k forks source link

Networking issue with 2.0.25 (and 2.1.0 Beta) and WFW 3.11 #2407

Closed brunocastello closed 2 years ago

brunocastello commented 3 years ago

Describe the issue I was using 2.0.22 with success in all my three VMs on my 2017 12.9 inch iPad Pro: DOS 622/WFW 311, W98SE, macOS9.2.2.

I decided to upgrade to latest version - 2.0.25. VMs all booted to black screen, to which I realized was a problem with my display device selection being show twice (settings screen and command line code, now fixed), so all VMs were booting normally again. I chose vmware-svga for both Windows now and I think macOS 9.2.2 is using VGA (not sure if it would have any benefit by choosing ATI-VGA, since G3/G4 PPC Macs had some ATI cards). Next, I was testing networking.

Windows For Workgroups 3.11 is using the AMD PCnet device, without absolutely any setting change, pretty default. Been using that configuration flawlessly since the first UTM version.

I then proceeded to downgrading UTM to 2.0.22, which was the version I was using before, and WFW 3.11 was able to connect to my network again. I decided to report this issue and stick to downgraded version (2.0.22) until a possible workaround is found for 2.0.25.

Configuration (required)

Crash log No crashes

**Opinion*** Did UTM upgrade any QEMU code or is using any nightly version? I tested QEMU 5.2.0 from homebrew with my macOS Big Sur, and WFW3.11 also booted and connected fine.

brunocastello commented 3 years ago

@osy while we're at it, I wanted to ask a question. I am looking for ways to enable audio for PowerPC Mac VMs (in special, the MacOS 9.2.2 VM I have).

The QEMU Screamer fork I am using on my Mac has a very good out of the box sound emulation following this guide: https://gist.github.com/openfirmware/a78141366efc81421228e40bd39d695d

I suppose that I could change the QEMU source here: https://github.com/utmapp/UTM/blob/master/patches/sources and recompile UTM myself? Would it work?

I think that it is still needed to pass a few parameters to compile (and use coreaudio on Mac, as per guide above mentioned) but no clue on how to do it for UTM. I know that @mcayland's version is outdated (last update 2018) but as of now whenever I want to run old MacOS VMs with sound such as OS9 and OS X Tiger, I have to use it instead of UTM.

osy commented 3 years ago

312 plans to add it but it's low priority right now as USB and other features are being worked on. You can try to build it yourself but if you don't have experience with that, it'll be difficult.

brunocastello commented 3 years ago

312 plans to add it but it's low priority right now as USB and other features are being worked on. You can try to build it yourself but if you don't have experience with that, it'll be difficult.

Thanks for the reply. I am currently testing MacOS 9.2.2 with UTM on my iPad Pro. I have set sound to use usb-audio, although most reports say it is unstable and does not work. After I booted, I tested the sound. Its working, at least for the beep it does when I scroll through the system menus. Then I went to the preferences and tested choosing different sound effects. Works great so far. So its usb-audio for OS9, then. You just cannot use more than 1GB RAM and for some newer OS X builds the sound just crackles. But these reports are as old as of 2018, surely QEMU has improved since then?

EDIT: usb-audio really still has issues. System randomly crashes either on boot or on shutdown. When it works, is amazing. So, its a hit-and-miss situation. I guess that my first shot at it was a lucky one.

EDIT 2: Yeah, usb-audio has partial functionality only. As soon as I load an app and start typing, the system hangs on. I'll turn it off for UTM and keep using the screamer fork when I want audio, until #312 gets worked up.

I will continue testing to see its stability. For now the system is still running. I think for OS X Tiger 10.4.11 it has to be different, as mentioned here:

-device ich9-usb-uhci1,id=newusb -device usb-audio,bus=newusb.0

I'll bring these tests to my macOS UTM install and try them out. For now I am still using 2.0.22 on my iPad due to the networking issue above mentioned. The macOS install is the latest, but I haven't tried it yet.

To sum up, it would be great for UTM if my tests are successfull, to let people know that you can emulate certain PowerPC Mac operating systems with sound and networking okay. That would give them a better choice than PearPC, SheepShaver or Basilisk II. Especially if you are on a M1 Mac. Cheers, mate, your UTM evolution has put me in a better emulation fun than I was two years ago.

brunocastello commented 3 years ago

I just updated to the latest UTM version for iOS and I am still getting that error for my network card (AMD PCNET) on Windows For Workgroups 3.11. Rolling back to UTM 2.0.22 fixes this issue. The following error appears when I load Windows:

"The following error occurred while loading protocol number 0.

Error 58: The network has responded incorrectly. . You may need to reconfigure the protocol using Network Setup. If your computer is a portable, make sure it is connected to its docking station."

Not sure why this happens for newer UTM versions after 2.0.22?

I noticed that Windows 98 SE detects a new Network Card and correctly loads the drivers and is still working, but WFW 3.11 has issues.

I had to roll back to 2.0.22 until this issue is resolved.

brunocastello commented 3 years ago

Tested again with 2.1.0 Beta.

Windows For Workgroups 3.11 is still unable to connect to the network due to the same problems mentioned above.

Note that on UTM version 2.0.22 the problem does not happen.

It doesn't happen either with the homebrew QEMU on my MacOS Big Sur, connection works there. Newer UTM versions broke the ability for WFW 3.11 to connect to the network.

brunocastello commented 3 years ago

@osy Fixed the issue by disabling the standard Network option offered by UTM and manually setting up the network configuration by adding the following parameter to the QEMU command line:

-net nic,model=pcnet -net user,hostfwd=tcp::80-:80,hostfwd=tcp::22-:22

Something clearly changed for network configuration in UTM between versions 2.0.22 and current beta. I'll close this issue since I have managed to fix it myself manually. The hostfwd parameters are not needed unless you want to use some other resources.

osy commented 3 years ago

Let’s not close it until it’s fixed for everyone. can you compare the generated arguments and your manual arguments (qemu -> export arguments) and see if there’s a difference?

brunocastello commented 3 years ago

@osy sure, I'll try to find it. Here it is:

UTM generated arguments

-device pcnet,mac=D2:7C:24:00:37:51,netdev=net0 -netdev user,id=net0

My custom arguments

-net nic,model=pcnet -net user,hostfwd=tcp::80-:80,hostfwd=tcp::22-:22

When disabling the UTM network to use my custom arguments, the UTM arguments for networking is removed and replaced by:

-nic none

Then I add mine at the end and hey presto it works on WFW 3.11.

Must note that the standard way on UTM without any custom configuration works normally for Windows 98SE, and both OS are using the same specified device (AMD PCNET) for networking.

osy commented 3 years ago

I wonder if the MAC address makes a difference. Can you add ,mac=D2:7C:24:00:37:51 to your command and see if it breaks it?

brunocastello commented 3 years ago

It returns invalid parameter "mac". I guess that I need to find out how to add the MAC address using the -net arguments instead of the -device arguments. Onto google search...

I could just try your original arguments without the MAC address instead. Would that help?

EDIT: I tried adding this way to my custom arguments:

-nic mac=D2:7C:24:00:37:51

And the same network error happened on Windows startup. It's definitely the MAC address setting.

brunocastello commented 3 years ago

I'll now try the UTM argument without the MAC address. Hang on...

EDIT: The same network error message. So it's not the MAC address, it's the way the device is configured in these arguments. I think it may be related to the DHCP.

Normally, WFW 3.11 should hang on a blue screen when the DHCP server does not respond before the desktop. But it is showing an error about the protocol:

"The following error occurred while loading protocol number 0. Error 58: The network has responded incorrectly. . You may need to reconfigure the protocol using Network Setup. If your computer is a portable, make sure it is connected to its docking station."

The legacy -net method works better than the -device and -netdev for WFW 3.11, while the newer method works best for newer Windows 9x and above.

mcayland commented 3 years ago

Just a few comments from my side as a QEMU developer (I see I was tagged earlier in the thread) which may help here.

As you may have noticed in recent QEMU versions there have been some changes in the the network options to remove some ambiguous behaviour around network device configuration, particularly around machines with in-built NICs.

The issue seems to be related to this:

-device pcnet,mac=D2:7C:24:00:37:51,netdev=net0 -netdev user,id=net0

In short, -device tells QEMU to plug the named device into the current machine which generally works except when the machine already has an in-built NIC. In this case you end up plugging a 2nd NIC into the machine and this is likely where the problems are coming from.

The proposed solution works but is not recommended:

-nic none -net nic,model=pcnet -net user,hostfwd=tcp::80-:80,hostfwd=tcp::22-:22

This is telling QEMU to not plug in the machine's in-built NIC but then plug in a new pcnet card into the machine in the next free slot which is wired up to a suitable default. According to the docs -net nic is deprecated and will be removed soon, so don't rely on using this.

Unfortunately without the full command line it's difficult to provide a solution but my guess from the above is that your QEMU machine already has an in-built NIC in which case you need to use -nic to wire the netdev to the in-built network device:

-nic user,model=pcnet,hostfwd=tcp::80-:80,hostfwd=tcp::22-:22

There have been discussions around exposing the netdev ids of in-built NICs so they can be wired in the same way using -netdev but currently this is still a work in progress.

As always do keep an eye on the QEMU documentation at https://qemu-project.gitlab.io/qemu/system/invocation.html for the latest updates in this area.

osy commented 3 years ago

Thanks for the explanation, I read that the -nic argument is obsolete: https://wiki.qemu.org/Documentation/Networking#The_legacy_-net_option

But there's behaviour there with built-in NIC that can only be accessed with -nic?

mcayland commented 3 years ago

Thanks for the explanation, I read that the -nic argument is obsolete: https://wiki.qemu.org/Documentation/Networking#The_legacy_-net_option

Be careful as these are actually 2 different options:

-nic is supported and used to wire up in-built network cards -net nic is deprecated and scheduled for removal soon

But there's behaviour there with built-in NIC that can only be accessed with -nic?

For the moment, yes. Currently for in-built devices you need to use -nic with the model/mac you wish to use and then append the configuration options that you would have used for the separate -netdev (this was deliberate to enable users to configure the in-built NIC in the same way).

As I mentioned above there have been discussions around auto-exposing the netdev ids of in-built NICs so that you can simply use -netdev anywhere but no-one has come up with a patch yet. The hard part is deciding which in-built ids to use and/or finding a way to expose them so that clients can construct an appropriate command line.

brunocastello commented 3 years ago

Hi, thanks for the great input! I'll definitely try it again.

I'll have to open 1 or 2 more issues later, after some tests last night, both for the PPC variant when running macOS 9.2:

How it is now

-device VGA

How it should be

-device VGA,edid=on

This change according to the e-maculation forums (and this QEMU documentation), provides some extra resolutions for macOS 9.x. I have no idea why it was working before without this parameter, but after the UTM update it suddenly stopped and I was booted to a 1920x1080 screen resolution.

the other -prom-env arguments mentioned on the website above linked, are optional and two of them are already used by default. The second one only shows a verbose boot if the user wants, commonly used to debug problems.

I'll open an issue for both things later when possible, unless osy or someone else points me to a workaround.

brunocastello commented 3 years ago

@osy @mcayland I can confirm that -nic user,model=pcnet,hostfwd=tcp::80-:80,hostfwd=tcp::22-:22 also worked flawlessly for me on WFW 3.11. -nic user,model=<user selected model> should be the new parameter from now on, for future versions or my case was just a very specific case?

Because the current parameter used by UTM actually works for Win 98SE and (AFAIK) macOS 9.2, and since no one complained yet, I assume that for other OSes too. Just not for Windows For Workgroups 3.11

mcayland commented 3 years ago

How it is now

-device VGA

How it should be

-device VGA,edid=on

This change according to the e-maculation forums (and this QEMU documentation), provides some extra resolutions for macOS 9.x. I have no idea why it was working before without this parameter, but after the UTM update it suddenly stopped and I was booted to a 1920x1080 screen resolution.

FWIW the default value of the edid parameter was changed from off to on nearly 2 years ago (see https://gitlab.com/qemu-project/qemu/-/commit/0a71966253c8b07586ebd6bee094a818e1e163b8) so either the QEMU version is old, or something else is amiss if -device VGA doesn't work on its own.

mcayland commented 3 years ago

@osy @mcayland I can confirm that -nic user,model=pcnet,hostfwd=tcp::80-:80,hostfwd=tcp::22-:22 also worked flawlessly for me on WFW 3.11. -nic user,model=<user selected model> should be the new parameter from now on, for future versions or my case was just a very specific case?

Great! As mentioned above, it's for the case of a NIC in-built to the machine. Without the full command line I can't identify the machine other than to say it is likely to be one of the pc-* or isapc machines.

Because the current parameter used by UTM actually works for Win 98SE and (AFAIK) macOS 9.2, and since no one complained yet, I assume that for other OSes too. Just not for Windows For Workgroups 3.11

Again as per my comment above: with the current parameter using -device you are effectively disabling the in-built NIC and then plugging a new NIC with -device into the next free ISA/PCI slot. My hunch would be that since WFW3.11 isn't plug an play, the OS is getting confused by the 2nd NIC - perhaps it doesn't support a 2 NIC configuration, or the new NIC is reusing the same IRQ as the old NIC but the routing is hard-coded for the on-board card. There are quite a few reasons that something could be wrong here.

brunocastello commented 3 years ago

@mcayland It's a pc-* machine, architecture i386. Emulating a basic early 90's PC with Pentium II, for playing games like Grand Prix 2, for example.

I think you are likely to be correct about your assumptions about WFW 3.11 and PnP devices, when Win98SE is using nearly the same machine and had no issues at all. So I agree with you and it was a very specific case, I think @osy does not need to make changes unless he thinks otherwise. Win98SE is using nearly the same configuration without any issue, networking works as expected with the original arguments.

FWIW the default value of the edid parameter was changed from off to on nearly 2 years ago (see https://gitlab.com/qemu-project/qemu/-/commit/0a71966253c8b07586ebd6bee094a818e1e163b8) so either the QEMU version is old, or something else is amiss if -device VGA doesn't work on its own.

Well, definitely something happened between 2.0.22 and 2.1.0 Beta. I am now using 1024x768x32 for macOS 9.2, but I'd rather be able to use 1280x960x32 because it looks best on my iPad Pro 12.9" screen. The 2.1.0 Beta is using QEMU 6.0.0, according to osy's documentation.

mcayland commented 3 years ago

@mcayland It's a pc-* machine, architecture i386. Emulating a basic early 90's PC with Pentium II, for playing games like Grand Prix 2, for example.

Ah nice!

I think you are likely to be correct about your assumptions about WFW 3.11 and PnP devices, when Win98SE is using nearly the same machine and had no issues at all. So I agree with you and it was a very specific case, I think @osy does not need to make changes unless he thinks otherwise. Win98SE is using nearly the same configuration without any issue, networking works as expected with the original arguments.

FWIW the default value of the edid parameter was changed from off to on nearly 2 years ago (see https://gitlab.com/qemu-project/qemu/-/commit/0a71966253c8b07586ebd6bee094a818e1e163b8) so either the QEMU version is old, or something else is amiss if -device VGA doesn't work on its own.

Well, definitely something happened between 2.0.22 and 2.1.0 Beta. I am now using 1024x768x32 for macOS 9.2, but I'd rather be able to use 1280x960x32 because it looks best on my iPad Pro 12.9" screen. The 2.1.0 Beta is using QEMU 6.0.0, according to osy's documentation.

That really helps being close to upstream. For the MacOS driver to offer extended resolutions then you need 2 things: 1) EDID must be enabled on the VGA device and 2) QEMU needs to be able to locate the special NDRV (native driver) binary which by default is located at $(PREFIXDIR)/share/qemu/qemu_vga.ndrv.

At startup the qemu_vga.ndrv is file is injected into the OpenBIOS device tree via a driver,AAPL,MacOS,PowerPC property where it is detected by MacOS and loaded instead of the default MacOS VGA driver - at which point it will decode the VGA device EDID block generated from the host with extended resolutions and make them visible in the Monitors control panel.

brunocastello commented 2 years ago

Hi, the networking problem popped up again when I tried latest macOS UTM version with my M1. When I attempted the same solutions we encountered here for previous versions, none of them worked. None of the other networking modes worked either.

Meanwhile, the vanilla QEMU 6.1.0 works flawlessly. Here's the command line I use to boot MSDOS 6.22 & Windows For Workgroups 3.11:

Screen Shot 2021-11-15 at 6 57 58 PM

brunocastello commented 2 years ago

I investigated a bit more, and noticed that UTM is setting the card to IRQ 11, while vanilla QEMU sets to IRQ 10. I can't figure out which device in UTM is attempting to use IRQ 10, to make a change and try to test if this was the culprit.

brunocastello commented 2 years ago

I fixed it. All I had to do was to set the Maximum Shared USB Devices to 0. Then disable the network interface. Then in QEMU tab I had to issue -nic user,model=pcnet,hostfwd=tcp::8080-:80,hostfwd=tcp::2222-:22 which was the original solution months ago by @mcayland.

I'll close this issue now that this was the culprit along the road since the start (IMO).