litchie / dospad

iDOS - DOSBox port on iOS
https://litchie.com/dospad
GNU General Public License v2.0
618 stars 133 forks source link

Games using mouse are unplayable on iPad with bluetooth mouse because of system circle cursor. #159

Closed jorycz closed 1 month ago

jorycz commented 2 months ago

Games using mouse are unplayable on iPad with bluetooth mouse because of system circle cursor, which shows notification center when reach top of the screen and dock when reach bottom of the screen. Both actions pause game and bottom edge sometimes close iDOS 3 and return to home. Mouse cursors (game arrow and system circle) are out of sync. Please see screenshot. iPad Pro 11" 2018. Latest iPadOS 17.6.1

UTM SE hides & sync cursors somehow, can something here in source help?

image

eduo commented 2 months ago

In particular, seems to be here: https://github.com/utmapp/UTM/blob/main/Platform/iOS/Display/VMDisplayMetalViewController%2BPointer.m

and

https://github.com/utmapp/UTM/blob/main/Platform/iOS/Display/VMCursor.m

jorycz commented 2 months ago

Thank you for sample sources correction.

eduo commented 2 months ago

Are you sure you're using iDOS 3? I've been testing a lot and I don't see the circle cursor at all.

I will test with Dune 2 which you have in your screenshot, since it may be specific to some mouse implementations.

jorycz commented 2 months ago

Yes, iDOS 3 current App Store version. It's not only in Dune 2 it's everywhere. Norton Commander and other games. That circle is iPadOS cursor. Are you testing on iPad with bluetooth mouse?

eduo commented 2 months ago

Yes, iDOS 3 current App Store version. It's not only in Dune 2 it's everywhere. Norton Commander and other games. That circle is iPadOS cursor. Are you testing on iPad with bluetooth mouse?

I tested with a trackpad and the pointer didn't show.

Then I hooked up a bluetooth mouse while the game was running, and the pointer appeared.

Then I killed the application and started the game again and the pointer no longer appeared.

Now I can't make the pointer appear no matter how I start the game and whether the mouse is connected or not when I start it up. I don't get a pointer when in DOS either. It seems to be properly masked. iPAD OS effects from the jmouse are also not visible (for example trying to pull the home bar, the three dots at the top or trying to move the mouse pointer to my mac). The native mouse is completely blocked from iDOS 3 except if I add a new one while the app is running.

Are you using the latest version from the app store?

jorycz commented 2 months ago

I am using latest release bought from App Store allowed by Apple recently. I made video with this issue https://youtu.be/wsIuQXuAEsE You can see on video, that I tried some things that sometimes help. For example, if I wait a bit (someone called me) and then returned to game, system cursor was gone and since then, all was OK for the rest of the run. Another time, when I switched to another app and returned to game, system cursor was also gone. But 95% of the time, it behaves like you can see on attached video. This time no switch or waiting (until system cursor fades) helped.

eduo commented 2 months ago

I have been testing this thoroughly and on an iPad Pro 11" with the latest iOS beta I can't see the circle pointer.

I loaded Discworld and the circle pointer appears a few seconds before iDOS takes over. The game cursor is visible the rest of the game. If I switch back and forth the circle is visible a couple of seconds and then goes away

jorycz commented 2 months ago

If I switch back and forth the circle is visible a couple of seconds and then goes away

This not happen to me unfortunately. Circle cursor reappears again when I move mouse as you can see in the video I posted in previous post.

Basically the video says it all.

Thank you for your time spent with testing.

eduo commented 2 months ago

I know this will sound like a stupid question, but in the video you don't ever click the mouse that I can see. You move it and then kill the app.

Unless I'm mistaken the mouse is captured after the first click and only then the circle disappears. Are you clicking at all and the circle persists? You mentioned you had used many games and NC, so I assumed you'd been clicking to test (when you click make sure to use the program's pointer and not the iPAD circle to target something).

jorycz commented 2 months ago

Question is ok but watch the whole video. I click the mouse several times. Mission was started by clicking for example.

mstrshikadance commented 2 months ago

I am having this same problem. Use to work perfectly, but since yesterday i get both cursors. In game cursor and also the IPad dot. I am using latest App Store version with iPad Pro and iPad Magic Keyboard/trackpad. iOS 18.1.

was playing under a killing moon for days without the issue..not i get both.

jorycz commented 2 months ago

I udpated one iPad to iPadOS 18 release released yesterday and issue still persist.

litchie commented 2 months ago

I am unable to reproduce this problem with iPadOS 17.6.1 and a Bluetooth mouse. But sounds like this is a problem that quite a few people are seeing.

jorycz commented 2 months ago

Today I connect USB-C mouse and issue is the same. So it's not related to bluetooth only. Also I noticed that iDOS 3 (updated to latest version) ignored mouse when connected first time. I need to (force) kill app and open again.

litchie commented 2 months ago

Upgraded iPad (9th gen) to iOS 18.0, still can't reproduce the problem.

eduo commented 2 months ago

Same here. I'm on ipad OS beta 18.1 beta 4 and still don't see it. It may be something very specific of some models? Specific of some peripherals?

jorycz commented 2 months ago

Guys do you have hardware keyboard? Issue seems to be related to HW keyboard. As soon as I don't have keyboard connected, mouse is working as expected.

As soon as I connect Smart Folio keyboard or even USB-C keyboard, system cursor appears and mouse issue is back.

When I disconnect keyboard and reopen iDOS 3 or restart iPad without HW keyboard, mouse is OK again.

jorycz commented 2 months ago

I think I found what exactly trigger this issue.

It is "Modifier Keys" mapping setting of the HW keyboard in iOS. You can find it in Settings > General > Keyboard > Hardware Keyboard > Modifier Keys. Here, you can map Escape key to some HW keys.

Here, If you set Caps Lock Key (or other key) action to Escape key, system cursor appear instantly and trigger this issue. As soon as you use Restore Default in Modifier Keys menu even with attached keyboard, mouse is behaving like expected (Turn off display/lock iPad and reconnect BT mouse to return to normal mouse behaviour).

Why I did it. After I installed iDOS app and tried some games, I found out that iPad Keyboard mapping is not working for Smart Folio Keyboard in app settings. For DOS games, ESC key is really basic key. So I map Esc to Caps Lock key in iOS HW keyboard settings.

Now I see 2 ways how to help people with Apple Smart Folio Keyboard. Fix system cursor (when something nasty happens) in case Escape is mapped to any HW keyboard key in iOS settings, or add mapping for this keyboard to iDOS settings so it's not necessary to remap it in iOS.

There are more layout versions so there is different "key picture" in upper left corner. Don't know if the scan code is the same or not though.

jorycz commented 2 months ago

Another thing I observed, maybe it helps with testing. When you hit CMD + . (period) system cursor appear and mouse stop working. This shortcut (cmd with period) is system Escape key emulation in iOS.

jorycz commented 2 months ago

This issue also triggers pressing Escape key on connected USB-C keyboard (standard DELL keyboard).

litchie commented 1 month ago

Hi Josef, that's very good findings. Thanks for digging into it. I will follow up on that clue.

litchie commented 1 month ago

I was able to reproduced the problem, and we should have a fix soon.

litchie commented 1 month ago

I have been trying the same hack UTM used, but it seems not very stable in iDOS. The system cursor still pops up now and then, especially when clicking. And the system dock sometimes would also pop out and mess things up. UTM doesn't seem to have this issue though. Still working on it, but will take more time than I expected.

jorycz commented 1 month ago

Fingers crossed! I can install ipa through Altstore in case some testing is needed. Can’t wait to finish my X-COM and Red Alert missions :)

litchie commented 1 month ago

You can try this build: https://testflight.apple.com/join/86LrYm5n

jorycz commented 1 month ago

Unfortunately almost same behavior. First I got this one https://youtu.be/BQds7j7uQe8 Then I reset modifier keys in Settings (so I lost ability to use Escape) and test it again. Mouse works well. Then I tried to use alt/option key as Escape key in mapping and got slightly different behavior https://youtu.be/zeF7hIHqs3Y but I could not be related. I noticed there is sometimes minor difference in cursor behavior if you run app when system cursor is showing and when not.

litchie commented 1 month ago

How about we map COMMAND+\ (Backslash) inside iDOS to escape?

jorycz commented 1 month ago

Would it be doable to add button to the top panel under settings gear icon, that would wait for any pressed key and map it to Escape key and store it (so you don't have to do that every time)? This would be universal solution and everyone with any layout can map any suitable key to Escape.

If not, your solution will work also of course.

My point is just to make it a little bit more universal with ability to setup Escape key to left top corner for any layout.

EDIT: Maybe it will be handy in future. If any hard coded key is needed for some game (for example right Control which is not on HW keyboard), you can easily add another mapping to make that game playable.

eduo commented 1 month ago

How about we map COMMAND+\ (Backslash) inside iDOS to escape?

I don't want to be that guy, but backslash is not easily available in all keyboards. In Spanish it's option-º so the shortcut becomes cmd-option-º .

Would it be possible to make it mappable? cmd-opt-. could also work as a default value since cmd-. is the standard cancel shortcut in iOS and MacOS and CMD, OPT and "." are available in all keyboard layouts.

jorycz commented 1 month ago

You can try this build: https://testflight.apple.com/join/86LrYm5n

I just tested both versions (AppStore and TestFlight) and I found behaviour of mouse in current AppStore version somewhat better.

litchie commented 1 month ago

@eduo How about COMMAND+, (COMMA) then? I would like to keep it simple if possible. In future, these would be configurable with per-game settings.

eduo commented 1 month ago

That's also pretty convenient (it's the shortcut for settings in mac applications but probably not an issue in iDOS)

litchie commented 1 month ago

@jorycz in the latest appstore version you should be able to use COMMAND+, for escape. That's what we can do for now.

jorycz commented 1 month ago

@litchie escape key mapping to CMD+, is working OK.

But sometimes (very often) it stays in loop. Check this https://youtu.be/kvQUnqUFkTs It’s the same behavior I described in my other issue #160 - sometimes it stays in loop and it seems like you are pressing the key again and again.

Should I open this issue or you want to track it in #160?

litchie commented 4 weeks ago

We can track it in #160.