search

GeyserMC / Geyser

A bridge/proxy allowing you to connect to Minecraft: Java Edition servers with Minecraft: Bedrock Edition.
https://geysermc.org
MIT License
4.69k stars 674 forks source link

Left click events not detected by other plugins when punching "air" - Geyser 1.16.220 #2113

Closed rayblon closed 1 year ago

rayblon commented 3 years ago

Describe the bug

When using the touch control schema on (probably) any bedrock client, tapping an area without an entity or block in the client's melee range will result in the 'air tap' being properly caught by Geyser (which will then render the hit cooldown as expected), but other plugins won't detect the left click.

To Reproduce

  1. Use a plugin that relies on left click events. I used Skript to send me a message whenever i performed a left click.
  2. Using a touch compatible device, tap anywhere that doesn't have an entity or block in melee range. The cooldown bar will appear, but the left click confirmation will not.

Expected behaviour

The left click with touch controls should be caught by plugins other than just Geyser, just like the controller and mouse&keyboard control schemas.

Screenshots / Videos

No response

Server Version and Plugins

ActionHealth, BKCommonLib, BottledExp, BungeeTabListPlus, BuycraftX, Citizens, CoreProtect, DeluxeChat, DiscordSRV, Essentials, EssentialsSpawn, FastAsyncWorldEdit (WorldEdit), floodgate-bukkit, Geyser-Spigot, goBrush, GriefPrevention, Guilds, HolographicDisplays, InventoryRollback, LibsDisguises, LightCleaner, LuckPerms, LWC, LWCTrust, MagicSpells, MMOItems, MobArena, MobArenaExtension, Multiverse-Core, Multiverse-NetherPortals, MundoSK, MythicLib, MythicMobs, OldCombatMechanics, PacketListenerApi, PerWorldInventory, PlaceholderAPI, PlotSquared, ProtocolLib, React, Shopkeepers, ShowItem, SkBee, Skellett, SkQuery, skRayFall, Skript, skript-reflect, skript-yaml, skUtilities, SkyChanger, spark, SuperVanish, TuSKe*, Vault, WarPlus, WorldGuard

Waterfall(Geyser & Floodgate are installed on the Waterfall bungee) Paper version 519

Geyser Dump

https://dump.geysermc.org/Z2qPhglFP9u82J2ahxoC5o0A0Pk79zPP

Geyser Version

1.2.1-SNAPSHOT (git-master-86b2901)

Minecraft: Bedrock Edition Version

Confirmed in 1.16.220 & 1.16.210

Additional Context

What we've tested

What isn't detected by other plugins: (on an android phone)

What IS detected by other plugins:

What we've discovered about touch control left clicking

  1. This behavior is not a product of software differences -- the control schemas behaved consistently between console, PC, and mobile, and the lack of 'air tap' functionality in touch controls was also consistent.
  2. Geyser detects the click just fine when tapping 'air' (you can see the hit cooldown bar when it happens)
  3. Skript doesn't detect 'air taps' targeting air when clicking with touch controls -- this is consistent with behavior observed with MagicSpells, and it's likely that no plugin other than Geyser is able to catch them at current.
  4. Tapping air with touch controls does not trigger the "punch" animation, while clicking with controller or mouse&keyboard does.
  5. This has nothing to do with the fact that clicking isnt bound to the reticle in touch controls, since it also doesn't work with split controls, which locks clicks to the reticle.
Camotoy commented 3 years ago

I have yet to look into this fully, but I can elaborate on one point:

Geyser detects the click just fine when tapping 'air' (you can see the hit cooldown bar when it happens)

This is specifically Bedrock sending a "sound event" that indicates it wants some form of an attack sound to be played - it was the workaround we devised to make the cooldown play in more scenarios. It doesn't mean that a packet is sent by Bedrock indicating what it's trying to hit. https://github.com/GeyserMC/Geyser/blob/master/connector/src/main/java/org/geysermc/connector/network/translators/bedrock/world/BedrockLevelSoundEventTranslator.java

rayblon commented 3 years ago

Hmm. I don't have the technical knowledge of packets to know what can and cant be done for this scenario, but would it be reasonably possible to extrapolate and construct the PlayerInteract event when a "sound packet" is sent without any accompanying data about what the client wishes to interact with?

It's not complete touch screen functionality, but it would achieve parity between touch controls and the other schemas wrt plugins, if it is possible.

Camotoy commented 3 years ago

We would also have to track if an interact event was sent recently, so it's probably not worth it to fix this tiny difference.

WillyWhitewool commented 3 years ago

I had a similar problem with Skript not detecting right click events with an empty hand UNLESS you were looking at a block. Also, holding an item made it work just fine too. Not sure if that information will help but thought I'd contribute.

Camotoy commented 3 years ago

I am able to confirm this - it is probably fixable, but would require timing the last sent arm swing as well, I believe, as the client sometimes intentionally does not send an arm swing packet.

chrismwiggs commented 2 years ago

Adding to this in favour of reclassifying it from low priority to something higher; this (in my opinion) seriously affects a wider range of plugins than the example of skript. It's a good way to quickly see that this event doesn't fire, but this also affects many other plugins that use the PlayerInteractEvent (RIGHT_CLICK_AIR) like ProjectKorra, Magic, some pets plugins, some plugins that use items as triggers (I've had this issue with Itemsadder items needing to be "clicked" on the ground for bedrock players) etc.

It gets used (for better or worse) fairly often and does seem important to feature parity! It's very confusing to new Bedrock players too; it appears as though things "sometimes" work because they don't realize that you have to target a block. Would go a long way in my eyes (and of course selfishly, my network's own use case of Projectkorra and ItemsAdder)!

Kas-tle commented 2 years ago

Posting a conversation from Discord with some potentially relevant info for completeness:

Kastle: Has anything changed since 1.16 that would make this less tedious to fix? Someone asked me about it so i figured I would relay https://github.com/GeyserMC/Geyser/issues/2113

Camotoy: There's a "workaround".

Kastle: Like the one described in the issue or?
Geyser hacks?

Camotoy: Something to do with the sound events Bedrock sends us.

Kastle: So the one mentioned in the issue then?

Camotoy: Yes.

Kastle: Oof

Redned: We just need to send the ServerboundSwingPacket to the server

Camotoy: I thought it was the UseItemPacket? Or both?
The swing packet I think is tough because doesn’t PC Bedrock swing?
Are you able to take a look later on?

Redned: for the air hit?
swing arm works fine in this case
iirc thats all java sends too
but... regardless it'll trigger everything needed for plugin devs, like the arm swing event & the playerinteractevent with the LEFT_CLICK_AIR action
HippieBeak commented 1 year ago

A previous workaround for this was to connect a controller or keyboard to one's mobile device... but as of the 1.19.50 update, this no longer works. Seems like some sort of regression...

chrismwiggs commented 1 year ago

Also experiencing this regression, which is unfortunate because of how long this has been an issue and that one workaround was all we could direct players to. Would love to see some conversation about this if nothing else!

wscott20 commented 1 year ago

I'm having a similar problem a different server. It's happening on an avatar bending server. When I try to use an ability that involves punching air it doesn't work.

Camotoy commented 1 year ago

See b5e1ddc3c86bd560e3ebbb8814c7d074c1bd6a69. Let us know if there's still an issue somewhere.