Switch controllers have random disconnects when pressing buttons rapidly

vaguerant commented 1 year ago

BlueRetro firmware version

1.8.4

BlueRetro firmware specification

HW1

BlueRetro firmware variant

Universal

BlueRetro hardware type

External adapter with detachable cord

Manufacturer

blue-retro.com (teamRe)

System used

Nintendo GameCube, Nintendo Wii EXT, none

Bluetooth controller brand & name

Nintendo Switch Pro Controller/Switch Online N64 Controller

What is problem? (only list ONE problem per report)

N.B. Before I begin, I'll note that I reflashed stock 1.8.4 to my BlueRetro adapter before confirming this issue. The log below is from my customized button layout firmware per #783, but nothing is dramatically changed and again, I confirmed this issue on stock firmware.

EDIT: When I originally reported this issue, I had only encountered it on GameCube and in homebrew software. However, through further testing, the issue occurs across all platforms and all software, so I'm editing this filing.

Pressing buttons and/or D-pad directions rapidly on the above-mentioned controllers causes them to disconnect after a very short period. I can usually trigger a disconnect within about 30 seconds of mashing any two buttons, less if I just mash everything all at once. This issue can be reproduced on all platforms and even no platform, by powering the BlueRetro over USB, connecting a Switch Pro Controller and mashing buttons until it disconnects.

Things which do not trigger the issue:

moving either/both analog sticks erratically
pressing and holding multiple/many buttons
leaving a controller connected for long periods without pressing any buttons

Only rapid pressing and releasing of the button inputs (D-pad, A, B, X, Y, L, R, ZL, ZR, Plus, Minus, Capture, Home) seem to trigger this issue.

These interruptions occur especially frequently in certain game genres such as retro twin-"stick" shooters like Super Smash TV for SNES. In this game, the D-pad and face buttons function as substitute joysticks, with constant movement between different walking and shooting D-pad and button presses.

I believe the reason I was originally encountering this issue entirely in homebrew software is that the apps I was using were mostly retro emulators, resulting in a lot of heavy D-pad and button usage, whereas the commercial GameCube (and Wii) software I was testing with primarily used the analog stick, which caused the issue to occur much less often, due to the reduced button (D-pad) usage. However, now that I'm able to reproduce the issue "at will" with some quick mashing, I can confirm that it is not in any way related to what software is (or isn't) running on the wired end of the adapter.

Below is the original issue report in case there's anything useful that remains in that info.

Old, incorrect issue filing

When running **homebrew** GameCube software, Switch controllers drop their connection fairly regularly (every ~5-10 minutes in normal use). The controller fully disconnects, with the LED blinking and shutting off and the BlueRetro blinking the Bluetooth LED if it was the last controller connected. This problem does not occur when running licensed, commercial software. The issue seems to vary across different homebrews/unlicensed software. Here's the behaviors I've observed so far: * **CubeSX (PlayStation emulator):** only NSO N64 loses connection, Switch Pro does not drop * **Genesis Plus GX (Mega Drive and other Sega emulator):** both controllers can lose connection * **Not64 (Nintendo 64 emulator):** both controllers can lose connection * **Swiss (all-in-one launcher):** neither controller loses connection * **Xeno Crisis (unlicensed game)**: both controllers can lose connection The issue only seems to occur if the controller is actively being used. Connecting a controller then leaving it sitting on a desk, they do not seem to drop connection. Talking with several homebrew devs, there's not any obvious connection between the apps which drop connection and those which don't--some use the older homebrew libogc library to talk to controllers, others use the newer libogc2. In the list above, all homebrews are using libogc2 except Genesis Plus GX, which is an original libogc app. Needless to say, actual GameCube controllers do not "lose connection" when running those apps. Speculating as an end-user, it seems like the way (some?) homebrews access GameCube controllers must differ in some way from the way retail software does. While actual GameCube controllers tolerate this, perhaps when passed over the BlueRetro, Switch controllers are not able to handle this behavior. libogc2 does allow setting the polling rate manually, but according to @Extrems (sorry for highlight), the default is unchanged. I'm not aware of any other ways that homebrew might behave differently to commercial software.

EDIT: Some additional info, I am aware that Switch controllers are prone to losing their connection when sent too many rumble packets or subcommands in a short period.

See SDL:

/* How often you can write rumble commands to the controller.
   If you send commands more frequently than this, you can turn off the controller
   in Bluetooth mode, or the motors can miss the command in USB mode.
 */
#define RUMBLE_WRITE_FREQUENCY_MS 30

https://github.com/libsdl-org/SDL/blob/d65861f0496a58b2c2642c249f03ac3a08e820c3/src/joystick/hidapi/SDL_hidapi_switch.c#L49-L53

And Linux:

/*
 * Sending subcommands and/or rumble data at too high a rate can cause bluetooth
 * controller disconnections.
 */
#define JC_INPUT_REPORT_MIN_DELTA   8
#define JC_INPUT_REPORT_MAX_DELTA   17
#define JC_SUBCMD_TX_OFFSET_MS      4
#define JC_SUBCMD_VALID_DELTA_REQ   3
#define JC_SUBCMD_RATE_MAX_ATTEMPTS 500
#define JC_SUBCMD_RATE_LIMITER_USB_MS   20
#define JC_SUBCMD_RATE_LIMITER_BT_MS    60
#define JC_SUBCMD_RATE_LIMITER_MS(ctlr) ((ctlr)->hdev->bus == BUS_USB ? JC_SUBCMD_RATE_LIMITER_USB_MS : JC_SUBCMD_RATE_LIMITER_BT_MS)

https://github.com/torvalds/linux/blob/9ed22ae6be817d7a3f5c15ca22cbc9d3963b481d/drivers/hid/hid-nintendo.c#L544-L555

What did you expect to happen?

Controllers should stay connected unless otherwise configured.

Attach files like logs or Bluetooth traces here

nso-n64-disconnect-log.txt

darthcloud commented 1 year ago

I'll have to check with a logic analyzer one of those homebrew output.

The only thing on top of my head would be polling rate that is too high and constantly interrupting the ESP32 which would affect the BT controller task.

vaguerant commented 1 year ago

I should have mentioned this in the original filing, but I don't think it's a broader Bluetooth issue because non-Switch controllers seem to be able to maintain their connections. Perhaps they're just more tolerant of lost Bluetooth packets or something, but I haven't been able to reproduce this issue with Wii Remotes and Extrems couldn't reproduce it with a Wii U Pro Controller, although that's really only "one" type of controller since WUPCs are really just Wiimotes in disguise. Still, that makes me think it must be something specific to the way Switch controllers work.

darthcloud commented 1 year ago

I took traces of the data line with gc bios, swiss and genesis+, they are all the same: 2 successive poll at 60Hz.

But I couldn't reproduce issue I played Sonic 2 15 minutes at lunch.

DragonKumera commented 1 year ago

Could it be similar to my issue? Though mine disconnect within like 1 minute and if not in use.

vaguerant commented 1 year ago

But I couldn't reproduce issue I played Sonic 2 15 minutes at lunch.

Hmm, odd. Perhaps it's specific to certain controller firmware versions? According to Joy-Con Tool, my Switch Pro Controller is on version 4.33 and my Switch Online N64 Controller is on 4.17.

vaguerant commented 11 months ago

Possibly notable: I just happened to be playing some GameCube homebrew stuff while the Switch controller was plugged into my Windows 10 PC. When the connection would drop, I would get the Windows disconnection/connection noises in quick succession. I don't know what exactly that means, but I wonder if the controller itself is completely rebooting, i.e. something causes it to crash rather than just losing its Bluetooth connection.

An additional note: when losing a connection in the conventional way (I just walked out the front door holding my controller and kept going until it dropped), the Switch Pro Controller goes back into searching mode, with the LEDs blinking from side to side. When a Switch Pro Controller gets disconnected from BlueRetro by whatever is causing this issue, the LEDs just shut off completely and only wake up again when the user starts pressing buttons, as if from cold boot. Again, this seems to suggest the controller is outright shutting down rather than "just" disconnecting.

While it may reduce controller responsiveness for certain functions, I wonder if some sort of subcommand queue might resolve the issue, where subcommands build up on the BlueRetro end and are sent out to the Switch controller at a predetermined rate rather than as soon as they're ready to go. I did try hacking around in the code a little to try to reduce the number of subcommands going out by commenting out these lines:

https://github.com/darthcloud/BlueRetro/blob/be6b1bd8debe818432e960231f7f6d424985bcee/main/adapter/wireless/sw.c#L530-L538

But while this (as expected) did cause rumble to stop working, the controller still lost connection after a few minutes of play. I can see that there's a lot more subcommand-related stuff in /main/bluetooth/hidp/sw.c, but that goes way over my head, I can't tell how often any of those subcommands are being sent or which can be safely disabled for testing purposes. If there's an easy way to reduce the frequency of subcommands being sent, I'd be happy to run off a test build to see if that helps.

EDIT: It's probably no more or less prone to this issue than any other GameCube homebrew, but I find this controller tester app useful for testing disconnections because you can see the buttons "freeze up" on the test screen as soon as the controller stops responding. By just mashing all major buttons at a reasonably fast rate (A, B, X, Y, L, R, ZL, ZR) I can usually get a disconnect within about 30 seconds.

So far, I am able to reproduce a disconnect by mashing any two or more buttons. Mashing a single button seems like it does not cause disconnects. Likewise, holding down multiple buttons does not cause disconnects. It seems like only pressing and releasing multiple buttons quickly crashes the controller.

I can't identify any pattern of specific buttons causing crashes when pressed in concert. I've tried L+R, A+B, X+Start and all combinations crash reasonably quickly. However, I did identify that manipulating only the two analog sticks does not cause a crash. It seems to be specifically connected to the buttons, not just inputs in general.

darthcloud commented 11 months ago

There is really no subcmd sent to the controller past the init in hidp. Only the rumble cmd which can be disabled by disabling rumble feature.

vaguerant commented 11 months ago

There is really no subcmd sent to the controller past the init in hidp. Only the rumble cmd which can be disabled by disabling rumble feature.

I see, thanks. This is definitely an odd one. In other news, with further testing I can confirm that this issue is not in any way related to the GameCube or to homebrew software. The issue occurs on all platforms regardless of the software that's running on the wired end of the adapter. I've updated the initial issue filing and will quote the new edited portion here for easier reading:

Pressing buttons and/or D-pad directions rapidly on the above-mentioned controllers causes them to disconnect after a very short period. I can usually trigger a disconnect within about 30 seconds of mashing any two buttons, less if I just mash everything all at once. This issue can be reproduced on all platforms and even no platform, by powering the BlueRetro over USB, connecting a Switch Pro Controller and mashing buttons until it disconnects.

Things which do not trigger the issue:

moving either/both analog sticks erratically

pressing and holding multiple/many buttons

leaving a controller connected for long periods without pressing any buttons

Only rapid pressing and releasing of the button inputs (D-pad, A, B, X, Y, L, R, ZL, ZR, Plus, Minus, Capture, Home) seem to trigger this issue.

These interruptions occur especially frequently in certain game genres such as retro twin-"stick" shooters like Super Smash TV for SNES. In this game, the D-pad and face buttons function as substitute joysticks, with constant movement between different walking and shooting D-pad and button presses.

I believe the reason I was originally encountering this issue entirely in homebrew software is that the apps I was using were mostly retro emulators, resulting in a lot of heavy D-pad and button usage, whereas the commercial GameCube (and Wii) software I was testing with primarily used the analog stick, which caused the issue to occur much less often, due to the reduced button (D-pad) usage. However, now that I'm able to reproduce the issue "at will" with some quick mashing, I can confirm that it is not in any way related to what software is (or isn't) running on the wired end of the adapter.

darthcloud commented 11 months ago

In the grand scheme of things Switch controller support on BlueRetro simply sucks in every aspect.

This isn't due to the way BlueRetro configs the controller, I did wireless trace of what a real Switch console do and pretty much do the same.

I think the real problem is the ESP32 BT controller that doesn't play well at a baseband level with those controller. It's closed source, so I need to go through espressif to get things fixed. That's a challenge on it's own since I've done a 100% custom BT stack.

My plan is to add support for an Infineon BT radio and then it will be easier to make comparison of two different BT controller using the same BlueRetro stack. I'm hopeful everything will work on the other and that this will rule out the BR stack, leaving the ESP32 BT controller to point finger at.

Using an ESP32 as a BT radio on windows/linux would be an interesting test too, to see if Switch controller behave as usual with that.

DragonKumera commented 11 months ago

That truly stinks. I am sure you can figure something out :)

bherdm commented 11 months ago

This morning I updated my N64 BlueRetro adapter to 1.9 and immediately started noticing these issues with the D-Pad causing disconnects on my N64 Switch controller. Googling the issue brought me here. Holding the button on the adapter for 10 seconds seems to have reverted it back to 1.8.3 and it's now working normally again.

vaguerant commented 11 months ago

Holding the button on the adapter for 10 seconds seems to have reverted it back to 1.8.3 and it's now working normally again.

Oh wow, fantastic observation. I tried dropping down to firmware 1.8.3 and I can confirm the issue is no longer reproducible on my end either.

darthcloud commented 11 months ago

That's good info, I did remove a old hack that was keeping the connection alive since I thought I had figured the proper way to do it. I will revisit that for v1.9.1 so that it their is at least no regression.

filthyfreak commented 11 months ago

I am also experiencing issues with my NSO N64 controller disconnecting if there are too many button presses within a short period of time. It is easily reproducible.

However whilst on 1.8.3 I was still having disconnection issues, but I cannot confirm if that was due to having many button presses.

To make it worse, whilst on 1.8.3, if my controller disconnected then it would refuse to re-pair unless I power-cycled the adapter/n64 console. This is a real pain because I'd be unable to save my game.

I will try downgrading to 1.8.3 to see if these issues reoccur.

EDIT: After reverting back to 1.8.3 the disconnection issue with too many button presses has disappeared. I am hoping the issue with being unable to re-pair without powercycling does not reappear but if it does then I will update this comment.

cyatta commented 10 months ago

Maybe related, but in https://github.com/darthcloud/BlueRetro/issues/824#issuecomment-1824045865 we’re finding more reliable connections with v1.7.3

edit: did have one drop out after pressing a couple buttons simultaneously and rapidly on 1.7.3 but it reconnected more quickly than newer versions I’ve tried

Xenphor commented 10 months ago

I just received my Blueretro n64 adapter and noticed drop outs when using a Switch Online N64 controller. I tried a dualshock 4 and it seems to work fine. I upgraded to 1.9 before using it.

darthcloud commented 10 months ago

I reproduced issues (pressing all buttons quickly to get disconnected)

I think this build fix it? https://github.com/darthcloud/BlueRetro/actions/runs/7081676820

Please try and report back.

vaguerant commented 10 months ago

This appears to resolve the issue on my end. I'm no longer able to trigger disconnects after mashing buttons for five full minutes, where previously I could reliably trigger a drop within a few seconds.

Tonight is busy but I'll make time to do a longer test tomorrow (~20 hours from now). I'm already pretty confident you've fixed this though. 👍

vaguerant commented 10 months ago

Was able to put in some more time, I did:

20 straight minutes of button mashing, zero issues except my fingers hurt
~20 minutes of Xeno Crisis (frantic 8-button game), zero issues
~40 minutes of Smash TV (" " "), zero issues

I'm happy to call this definitively resolved. Thanks so much for all the work you've put into Switch, I know it's been a nightmare on your end. I won't close this issue purely in case you want to keep it open until the next release is out, but obviously feel free to close per whatever is your standard practice. Thanks again and great work!

darthcloud commented 10 months ago

@vaguerant thanks for testing!

I did test also last night but on N64 with RetroTime dongle, played with my everdrive using an NSO N64 ctrl for around 1h30 with no issues whatsoever. Swaping often between rumble and controller pak. Look more solid that it ever was TBH.

darthcloud commented 9 months ago

Fix is now in v1.9.1

darthcloud / BlueRetro