Build variant which uses TCG interpreter

[osy]

From: https://github.com/utmapp/UTM/issues/397#issuecomment-690932722

For iOS 14 non-jailbroken we currently require Xcode tethered to work. However, if we use tcgi, which does not require JIT/dynamic-codesign then it would still work but slowly. However, it may still be fast enough for some workflows.

Since QEMU only supports tcgi as a build-time option, we would need to either build two variants of UTM (one IPA for jailbroken and one IPA for no-JIT) or alternatively we can include two copies of the QEMU dylibs in UTM. This would double the size of UTM from ~200MB to ~400MB.

[brunocastello]

I'd vote to double the size until a definite solution is found, I believe this is less disruptive to the workflow. If the devs can find a solution for non-jailbroken, excellent and is still only one IPA. But if it's not possible and a jailbreak for iOS 14 is released in the meantime, then switch to a two different IPAs or just stick with the jailbroken option. Just my opinion. I'd jailbreak just for UTM on my iPad Pro, currently on iOS 13.5 running previous UTM build.

EDIT: The iPad Pro is currently on iOS 13.6.1, so no fun for me too. Sticking to previous build for now.

Looking forward to trying the version for non-jailbroken on my iPhone X, the only device I have on iOS 14.

[saagarjha]

+1 on two IPAs. Or perhaps you can download the right set of libraries in-app?

[osy]

Is it possible to dlopen outside of the .app bundle?

[saagarjha]

No idea. I was thinking that this should work from a library validation perspective since Apple isn't resigning our app…although I guess everyone is resigning it themselves so this may not actually work in this case either.

[asdfugil]

Also +1 on 2 ipa. Maybe we can consider other solutions later, but this is the most simple way to get started while not being bloat.

[iamrecursion]

I'd also add my +1 to two separate bundles. No reason to cause people to download bloat for something they can't currently use (in the non-JB case) or won't use (in the JB case).

[brunocastello]

I am sorry to interrupt, but I know an app that is powered by BOCHS that works from iOS 10 to iOS 14 with no Jailbreak, it's called, "iBox v2.6.8" (you can download the IPA from here and here is their project) you can get some inspiration and get some ideas from iBox v2.6.8 to make UTM work in iOS 14 without Jb, I know you can use TCG, but I am just giving you ideas to make UTM work with iOS 14 without Jailbreak, if I interrupt, then I will delete my comment.

I understand your intention here, good intentions, but I think you have no idea of what is the problem here. Not even the technical implications. I'll just sum it up and say that BOCHS is nothing like QEMU and is much more closer to DOSBox and PCem than QEMU; and iBox is based on old, abandoned and deprecated iOS code (last update was 2018), so nothing can be used from there.

While I'm at it, I would like to add +1 to 2 separated IPAs. I made a different suggestion before because I thought from a developer perspective it would be better, but for the long run, 2 separated builds make sense. Although there are reports of working iOS 14 jailbreaks, no public tools or techniques for jaibreaking were released yet, no idea when they will be.

[obbcth]

I wonder if tcgi build can be uploaded on Testflight.

[itselijahciali]

While Bochs is deprecated and old, I do think considering other backends might be an interesting idea for the non-JB release. Have we tested KVM (not sure that could be ported at all) or something like that?

[osy]

KVM depends on Linux kernel features and HW virtualization features. Both missing from iOS and Apple A chips.

[saagarjha]

A14 has hardware virtualization, although you would likely need a jailbreak to access it.

[osy]

I'm of course referring only to things that are released

[professorUnknown]

Anyways Apple probably locked virtualization in iBoot so kernel exploits won't work.

[rylyguy]

Once macOS Big Sur releases would it be possible to somehow hackintosh it to run on a raspi or something similar as a debugserver or potentially altserver.

[osy]

I’m not sure how that’s relevant to this issue?

[rylyguy]

If it could serve as a debug server it could go in a small enclosure that could just sit in a backpack or pocket and could be used to launch utm on non-jb devices

[brunocastello]

Forget that idea, it will never happen. Hackintosh will be dead in two years when Apple stops supporting Intel Macs, stops distribution of Intel versions of their OS and kills Rosetta 2.

Apple Silicon Big Sur version will never run natively on anything different, not only because of drivers and dependencies, but because the Apple Silicon is a SoC with other components that do not exist in other ARM computers, like the T2 chip, Neural Engine, its own custom integrated GPU... its not just an “ARM chip”, it has its own structure and instruction sets based on the ones that were licensed to Apple perpetually.

Both OS and Apple Silicon SoC will be locked down and tied together. Take the words of Craig Federighi from WWDC 2020, it will not run dual boot with any other OS, he stated that virtualization is the way to go for other OS inside the Apple Silicon Macs.

[brunocastello]

Anyways Apple probably locked virtualization in iBoot so kernel exploits won't work.

True, but we will only know for sure when the AS Macs are released and when we compare them to the latest iPads with the same SoC variant. Its known that they will have the same SoC just different variants, like A12X and A12Z today.

[osy]

I got TCGI working but bad news it’s super unstable. I tried QEMU with tcgi on my Mac just in case and get the same behaviour. Can’t load into Windows 98 setup. Can’t load Windows XP either without BSOD. Seems like QEMU devs never intended tcgi to be used for real, it’s only for debugging and can only load really basic examples. I’m not sure if it’s even worth releasing.

[saagarjha]

This matches my experience, then. I thought I was doing something wrong when my Linux panicked nearly every time I tried to boot it…

[brunocastello]

My experience with QEMU on macOS without HVF was about the same speed I get on my iPad Pro 2nd gen, when I used Windows 98 SE. No crashes, on both macOS (without HVF) and iPad.

When I tried to use HVF, for some reason I can’t use with qemu-system-i386, only the x86_64 variant. Fine, I tried that, turns out that it will crash, yes. So apparently Windows 98 does not like HVF. In the last two months I have been extensively testing QEMU as an alternative on macOS, I have built VMs with XP and 7 and HVF acceleration enabled. I cant remember if I had any problem. I just miss 3d acceleration for games.

To sum up my post, basically I dont have the same experience as you guys. Mine was somewhat better. The same Win98SE image I did on iPad with UTM was used on macOS and QEMU from homebrew (5.1.0). Runs fine, no HVF, no acceleration.

[saagarjha]

To be clear, we are talking about the TCG interpreter. Perhaps you were using no-hypervisor-but-JIT-enabled TCG?

[brunocastello]

Well, can you tell me how do I check if its JIT enabled? I know for sure that its TCG, because I did not use accel HVF with my parameters for Win98SE. I can share them tomorrow when I get out of a bloody blackout, I am without power for 26 hours so far. And the parameters are in a file stored on my Time Capsule so no way to access them now.

[osy]

HVF is a different thing, it’s akin to KVM for OSX. We’re talking about TCGI, the only way to enable it is to build QEMU with a special build flag, you can’t enable it otherwise.

[brunocastello]

HVF is a different thing, it’s akin to KVM for OSX. We’re talking about TCGI, the only way to enable it is to build QEMU with a special build flag, you can’t enable it otherwise.

Yeah, I know, I have tried them both (HVF on macOS and KVM on Debian). What I was trying to say here is that I have used Windows 98 SE on QEMU without any acceleration, just TCG, and I had a little bit faster VM than my iPad Pro 2nd gen. My Mac is a 2013 13-inch retina MacBook Pro, i7 2.8 GHz and 16GB RAM. I used the same image I created on UTM. Apparently its qcow2 format.

XP and 7 however does need acceleration, its bloody slow on my mac without it. While XP on my iPad wasn’t THAT bad and slightly better on my iPhone X.

[saagarjha]

The TCI implementation claims that they have been able to boot things. I am unsure why we're seeing it be very broken; perhaps it's bitrotted since then? In any case there's a nice instruction-stepper that you can apparently use for testing, perhaps it might shed some light on what's going on.

just TCG

The thing is that TCG can either go through the JIT or the interpreter, and we're not sure which one you're using. Based on your observed performance I'd guess you're using the JIT, since it is also enabled by default if you run the configure script (even if you turn off HVF support).

there is a raspberry pi alternative that you can hackintosh

Not really relevant here, would you mind taking this somewhere else @IIWare?

[saagarjha]

I would mark it as off-topic at the very least.

[brunocastello]

The thing is that TCG can either go through the JIT or the interpreter, and we're not sure which one you're using. Based on your observed performance I'd guess you're using the JIT, since it is also enabled by default if you run the configure script (even if you turn off HVF support).

Damn, I probably had it then. For the macOS version of UTM this isn’t a problem. The Apple Silicon does have a hypervisor, it’s documented, but it will only work for ARM based OS AFAIK. So I’d say that TCG with/without JIT or something else is the only way to go from here and the future.

Checkra1n has jailbroken iOS14 for A9 devices a few hours ago, shouldn’t be a long time to have it for newer devices. As for non jailbroken, well... how about involving/inviting some of the jailbreak devs in the conversation? So we could try to explore the possibility for another exploit to try and run it with JIT?

[osy]

For the context of this issue, I think I’m going to drop work on the TCGI branch since it’ll require significant investment on the QEMU side.

[iamrecursion]

Disappointing but understandable!

[brunocastello]

The 41 hour blackout finally ended after a lawsuit from my condo against the power company. I have power and internet again. Meanwhile, closing this issue means that the things (at least for now) stands as they are today (requires jailbreak to work on iOS 14)? My iPad Pro is on 13.6.1 (I think), can I install the latest IPA on it to try it out?

@saagarjha do you still want my file with the parameters to run Win 98 SE ?

[osy]

Yes jailbreak or Xcode tethering. You can install latest ipa on 13.6.1.

[osy]

FYI after reading https://github.com/qemu/qemu/tree/master/tcg/tci closer, if you look at the compatibility table it appears that booting Linux in system emulation mode has never worked. GRUB loading is as far as it goes and that matches my experience as well. Seems like it is more useful for user level emulation.

[conath]

Does this news about the TCI seemingly having gotten much more stable recently change the status of this issue? There's a fork of UTM with TCI enabled but it doesn't build for me, requires a sysroot with a dylib that isn't included.

[osy]

You need to grab a sysroot from a "dev" branch build.

[conath]

I tried building with the sysroot from this action run but it includes libffi.6.utm.dylib while the fork tries to link against libffi.7.utm.dylib.

[osy]

Ah I didn’t realize there’s a lot of changes in the dependencies. You can just fork the fork and get Github actions to build it for you.

[asdfugil]

This can be closed as fixed.