Reboot Hydro Platinum device when reads fail

[EvanMulawski]

From #76, #78, #127, #130

When a Hydro Platinum device read fails multiple times, iCUE reboots the device:

https://forum.corsair.com/forums/topic/176494-corsair-h100i-platinum-se-randomly-stops-and-restarts/ https://forum.corsair.com/forums/topic/171275-hydro-h100i-connectingdisconnecting/

Implement reboot logic when a read fails by sending the reboot packet.

Notes:

• Based on an observation in #78, the device accepts writes and adjusts pump/fan speeds accordingly even when this issue occurs (implies only reads fail).

[EvanMulawski]

@jflevesque @Profetorum @BrendanMan3 @larz101 Please try the attached build. This will attempt to reboot the Hydro Platinum device if reads start failing (just like iCUE does). If the device reboots, the latest CorsairLink.log file will have "Rebooting device" followed by a read of the device state. Thanks!

FanControl.CorsairLink-HydroPlatinumReboot-v20231223r001.zip

[BrendanMan3]

CorsairLink.log

[EvanMulawski]

@BrendanMan3 Do you have any other software running that is communicating with the device (iCUE, Corsair Service, OpenRGB, SignalRGB, monitoring software, etc.)?

[BrendanMan3]

CorsairLink.log log.txt Corsair looks to all have been shut down. The logs I just sent were after running fine for a few minutes.

I was running signal RGB. I'm testing now with it disabled.

[BrendanMan3]

I can run SignalRGB with the H115i disabled and fan control has not (yet) thrown any errors.

[Profetorum]

@jflevesque @Profetorum @BrendanMan3 @larz101 Please try the attached build. This will attempt to reboot the Hydro Platinum device if reads start failing (just like iCUE does). If the device reboots, the latest CorsairLink.log file will have "Rebooting device" followed by a read of the device state. Thanks!

FanControl.CorsairLink-HydroPlatinumReboot-v20231223r001.zip

CorsairLink.log Still erroring out. I think it actually rebooted the pump or something, since the pump lights blinked, but apparently it didn't do much. Only using hwinfo, no other conflicting softwares afaik

[EvanMulawski]

@BrendanMan3 What version of SignalRGB? Compatibility with Hydro Platinum devices requires SignalRGB 2.3.45-beta or later.

[EvanMulawski]

@Profetorum Thanks. I think I need to close and reopen the device after rebooting - will have another build soon.

[EvanMulawski]

New build.

FanControl.CorsairLink-HydroPlatinumReboot-v20231224r001.zip

[Profetorum]

New build.

FanControl.CorsairLink-HydroPlatinumReboot-v20231224r001.zip

Going to download it. Not sure when i'll report back though since tomorrow i won't be much at my pc, and i can't force-reproduce the issue (Merry christmas btw :) )

[BrendanMan3]

I'll give the new build a try.

I am using SignalRGB 2.3.47-beta.

So far no errors on build from yesterday as long as H115i is disabled in SignalRGB. As soon as it is enabled in SignalRGB I start to get errors and when I restart fan control it doesn't detect any of the corsair products.

[Profetorum]

New build.

FanControl.CorsairLink-HydroPlatinumReboot-v20231224r001.zip

CorsairLink.log Ok it was surprisingly quick!

Errored out again - can't tell if it restarted the pump, haven't noticed

[EvanMulawski]

New build.

FanControl.CorsairLink-HydroPlatinumReboot-v20231224r002.zip

[BrendanMan3]

Errors detected using 24r002 build while the H115i is enabled in SignalRGB - Will test with it disabled. CorsairLink.log

[BrendanMan3]

With the H115i disabled in Signal RGB I have not had any issues after 6 hours of running.

[EvanMulawski]

New build. Still trying to successfully reboot the device.

FanControl.CorsairLink-HydroPlatinumReboot-v20231227r001.zip

[Profetorum]

Going to download this one as well. Haven't had a pump failure with the previous build, it never restarted. So I can't tell if it was working.

Going to report back next time it tries to restart the pump

[chjohans]

I too get errors with my H115i when running SignalRGB, the latest beta v 2.3.47-beta. If I disable the H115i in SignalRGB everything works just fine. I'm running HWiNFO64 as well, and it too looses the H115i sensore readings when this happens. This makes me question if the CorsairLink mutex implementation for Corsair Pro/Platinium AiOs they supposedly implemented in v 2.3.45-beta is working as intended: https://docs.signalrgb.com/changelogs/2-3-45

So I don't think the issue is with this plugin.

I've not had much luck in communicating with SignalRGB support in the past (I'm only a "free" user).

If you @EvanMulawski has communicated with them re the CorsairLink Mutex in the past then maybe you have a better channel to give them feedback on this?

[EvanMulawski]

@chjohans Can you attach your latest CorsairLink.log file? I'm thinking it could just be an inherent issue with the AIO's microcontroller and/or firmware - which is why iCUE has logic built-in to reboot these devices when it can no longer read from them.

[EvanMulawski]

I just ordered an H150i RGB PRO XT to test and troubleshoot this locally - will provide a status update after it arrives.

[chjohans]

Wow, thanks a million @EvanMulawski for your commitment to this, which is nothing short of impressive. I'll buy you a cup of coffee or two again shortly, and attach my logs as soon as I return home in a few hours.

[Meh7991]

I just installed the latest release of this plugin, and updated SignalRGB to 2.3.47-beta to see if I would have any luck with my H150i RGB Pro XT but nope. SRGB constantly logs "CRC error from color packet", as it used to for me when I was having issues.

Exiting FanControl stops those errors being logged in SRGB.

I have set my lighting effects to have a dim white background colour, so as a workaround for the time being I have just set my H150i to a static dim white colour in iCUE.

As you have ordered a the same cooler that I have, @EvanMulawski, I'll be willing to share logs or anything again if you need it.

[chjohans]

Running HWiNFO64 with "CorsairLink and Asetek Support" enabled and Fan Control with your plugin in parallel works just fine, so it seems like the shared mutex is doing its job when used correctly. I can see that HWiNFO sometimes loses Corsair readings for a very brief moment, probably because the mutex is locked during its update cycle, and that is not unexpected.

But, when I enable my "Corsair Hydro H115i RGB PRO XT" in SingnalRGB (v2.3.47-beta) then HWiNFO immediately loses all reading from this device, and FanControl starts to complain about errors as well. Please see the attached CorsairLink.log. Nothing is logged in log.txt

When this happens I have to reboot my computer to get my AiO back in HWiNFO and FanControl again. Just stopping SignalRGB and restarting Fan Control and HWiNFO makes no difference.

CorsairLink.log

[EvanMulawski]

I was able to reproduce this after a few minutes of testing - currently investigating.

[EvanMulawski]

New build. This will successfully reboot the device when necessary to restore Fan Control and SignalRGB connectivity.

FanControl.CorsairLink-HydroPlatinumReboot-v20240105r001.zip

Testing notes:

• I ran a USB capture while both SignalRGB and Fan Control were running concurrently. It appears that version 2.3.47-beta+ef6c0587 does use the well-known mutex correctly. I also confirmed this by monitoring the mutex when only SignalRGB was running.

• SignalRGB reports about a 6fps loss while Fan Control is running and the frame times spike every second as it waits to reacquire the mutex. (I was seeing 21fps without Fan Control running and 15fps with both running.)

• With only SignalRGB running (i.e. Fan Control not running concurrently), the device still got into a bad state after a short period of time. (Launching Fan Control successfully rebooted the device and both applications recovered.)

---

@Heal-Bot An observation from the packet capture:

When SignalRGB acquires the mutex, it should read the state immediately. Byte index 1 will be the previous sequence number - use this to generate the next sequence numbers.

Additionally, you may want to consider ignoring the responses from all lighting and cooling commands - I have found these to be unreliable and they frequently fail CRC verification. The read operation itself isn't necessary and was removed from OpenRGB.

[Meh7991]

v20240105r001 doesn't seem any different for me.

I'm running FanControl v177, and SignalRGB 2.3.47-beta+ef6c0587

Here are my logs (with FANCONTROL_CORSAIRLINK_DEBUG_LOGGING_ENABLED enabled) - ~10 minutes: CorsairLink.zip The device started bugging out after around 2 minutes of being enabled.

Here are my logs (with FANCONTROL_CORSAIRLINK_DEBUG_LOGGING_ENABLED disabled) - ~10 minutes: CorsairLink.zip The device seemed more stable this time, lighting effects were still a bit laggy but it began bugging out after ~6 minutes of being enabled (instead of displaying a static blue colour as below, it started flickering in rainbow colours towards the centre).

Here is a physical picture of my cooler, as you can see in my below screenshot I have the 'Fireworks' effect set - but this is what my cooler ends up doing eventually: That blue colour is the 'shutdown' effect I have set in SignalRGB, not sure if that has any relevance.

Here is a screenshot of the device's log in SignalRGB:

I didn't see the framerate change at all from 15fps.

Let me know if you need any more information. Happy to test out different builds.

[chjohans]

The attached version worked a bit better. I could now start SignalRGN and enable my H115i without FanControl instantly showing error messages.

But after a few minutes of playing around with SignalRGB the same error came back, and just as before the only way to get FanControl to work again is a full reboot. If I just restart FanControl (after stopping SIgnalRGB) it will not initialize, eg it will not find any Corsair sensors.

Please see the attached log file. Do take note that it does not contain logging from the restart attempt after the error happened. If ytou need that as well please let me know.

CorsairLink.15.log

[EvanMulawski]

@Meh7991 @chjohans Can you run this build without SignalRGB running and let me know if you encounter any issues? Also, please try running HWiNFO64 (with Corsair support enabled) and this build of OpenRGB concurrently. Thanks!

[Profetorum]

Sorry guys I might be wrong here, not at my PC right now, but iirc the hwinfo says it should be disabled in order to skip polling for Corsair devices? Can someone check

[EvanMulawski]

Sorry guys I might be wrong here, not at my PC right now, but iirc the hwinfo says it should be disabled in order to skip polling for Corsair devices? Can someone check

Only if iCUE or other applications that don't use the Corsair mutex/lock are running.

[chjohans]

@Meh7991 @chjohans Can you run this build without SignalRGB running and let me know if you encounter any issues? Also, please try running HWiNFO64 (with Corsair support enabled) and this build of OpenRGB concurrently. Thanks!

I'm always running HWiNFO (with Corsair support enabled) :=)

I have been running this build together with HWiNFO with no issues since my post above. So without SIgnalRGB running it seems just fine.

I have been running the build of OpenRGB you are linking too together with HWiNFO and your FC plugin before with no issues, but I'll do some more testing with that and this build of your plugin and report back.

[Meh7991]

@Meh7991 @chjohans Can you run this build without SignalRGB running and let me know if you encounter any issues? Also, please try running HWiNFO64 (with Corsair support enabled) and this build of OpenRGB concurrently. Thanks!

Seems to be no issues for me running this setup.

[chjohans]

@Meh7991 @chjohans Can you run this build without SignalRGB running and let me know if you encounter any issues? Also, please try running HWiNFO64 (with Corsair support enabled) and this build of OpenRGB concurrently. Thanks!

I'm always running HWiNFO (with Corsair support enabled) :=)

I have been running this build together with HWiNFO with no issues since my post above. So without SIgnalRGB running it seems just fine.

I have been running the build of OpenRGB you are linking too together with HWiNFO and your FC plugin before with no issues, but I'll do some more testing with that and this build of your plugin and report back.

Been playing around with all kinds of effects in OpenRGB for the past hour or so, no issues at all. I was running HwiNFO and FanCOntrol at the same time of course.

[BrendanMan3]

Successfully rebooting the H115i when it fails and is enabled in SignalRGB. However it is occurring way too frequently and is logging endlessly. CorsairLink.11.log

[EvanMulawski]

New build.

FanControl.CorsairLink-HydroPlatinumReboot-v20240112r001.zip

[Meh7991]

New build.

FanControl.CorsairLink-HydroPlatinumReboot-v20240112r001.zip

This build looked to be working perfectly with SignalRGB running, and the H150i enabled in it. I thought to myself, "omg finally, everything is synced!!".

Five minutes or so later:

Here is my log, might be a bit large because I forgot to delete it before running the new build: CorsairLink.zip

I had to fully power cycle my PC in order for my H150i to become recognisable again in FanControl.

Looks to be getting close to being stable though! The lighting effects looked much smoother than with the v20240105r001 build.

[EvanMulawski]

I am making some changes to the SignalRGB Hydro Platinum plugin code and will post the modified version after testing (with instructions).

[EvanMulawski]

Updated SignalRGB plugin attached for SignalRGB 2.3.45-beta and above.

signalrgb_hydro_platinum_plugin_v20240112r001.zip

Instructions

1. Unblock the downloaded ZIP file. (Properties > Unblock > OK)
2. Exit SignalRGB.
3. Open the directory C:\Users\<username>\AppData\Local\VortxEngine\app-<version>\Signal-x64\Plugins\Corsair\Platinum_Pro_XT_AIO
4. Copy the existing Platinum_Pro_XT_Elite_AIO.js file to another location to create a backup (do not create a copy in this directory).
5. Replace the file with the attached file.
6. Launch SignalRGB.

[Meh7991]

Updated SignalRGB plugin attached for SignalRGB 2.3.45-beta and above.

signalrgb_hydro_platinum_plugin_v20240112r001.zip

Instructions

1. Unblock the downloaded ZIP file. (Properties > Unblock > OK)
2. Exit SignalRGB.
3. Open the directory C:\Users\<username>\AppData\Local\VortxEngine\app-<version>\Signal-x64\Plugins\Corsair\Platinum_Pro_XT_AIO
4. Copy the existing Platinum_Pro_XT_Elite_AIO.js file to another location to create a backup (do not create a copy in this directory).
5. Replace the file with the attached file.
6. Launch SignalRGB.

Using your CorsairLink plugin v20240112r001, SignalRGB v2.3.47-beta+ef6c0587, and your attached SignalRGB plugin, everything seems to be working perfectly!

I will leave this running for the next several hours and report any findings.

[Meh7991]

After about 1 hour of running, FanControl popped up with an error alert. CorsairLink.zip

[chjohans]

My results were a bit different, no issues so far after running this version for about 8 hours with SignalRGB effects enabled all the time. SignaRGB v2.3.47-beta, the latest version of FanControl with this plugin, and HWiNFO with "CorsairLink and Asetek Support" enabled. No errors were observed and no errors in any log files so this seems to be working for me and my H115i.

[EvanMulawski]

After about 1 hour of running, FanControl popped up with an error alert. CorsairLink.zip

Did the device stop functioning after those errors? The checksum errors are typically harmless but it is interesting that 10 occurred in a row.

---

My results were a bit different, no issues so far after running this version for about 8 hours with SignalRGB effects enabled all the time. SignaRGB v2.3.47-beta, the latest version of FanControl with this plugin, and HWiNFO with "CorsairLink and Asetek Support" enabled. No errors were observed and no errors in any log files so this seems to be working for me and my H115i.

I experienced the same thing with my H150i PRO XT - no errors requiring a device reboot and only sporadic checksum errors in the log after 6 hours of testing.

[Meh7991]

After about 1 hour of running, FanControl popped up with an error alert. CorsairLink.zip

Did the device stop functioning after those errors? The checksum errors are typically harmless but it is interesting that 10 occurred in a row.

I will re-install these plugins and see if that was just a one-off glitch. When I saw that error pop up earlier, I basically immediately quit FanControl and SignalRGB and reverted to the working plugins, all has been fine ever since (apart from the lack of synced lighting of course haha). But, if I remember correctly, I swear I saw the fan speeds for the cooler say 0RPM in FC - could be wrong about that, will see if/when it errors again.

I did not need to power cycle my system for FC to recognise the cooler/fans again, however.

Edit: after running for two hours, I haven't had any FC errors. My CorsairLink.log has 5 checksum failures logged, but all is good. I will continue running again tomorrow for as long as I can and report back then.

[Meh7991]

After about 1 hour of running, FanControl popped up with an error alert. CorsairLink.zip

Looks like this was a one off glitch. I have had the new plugins running for almost 10 hours straight now and there hasn't been any errors. I'll attach my log file, which also includes ~2 hours from yesterday evening. CorsairLink.zip

If I come across any errors or anything, I will be sure to update here, otherwise this seems as stable as it'll get! Good job with with everything, @EvanMulawski!

[Meh7991]

Just got the error again after almost 2 hours of running today.

It seems that I get the error whenever the 'Checksum failure' error occurs 10 times in a row. CorsairLink.zip

Everything is still working though, FC still updates fan speeds and all the lighting works in SRGB. No errors in SRGB console.

[chjohans]

My PC is "always on", but goes to sleep when not being used. Attaching the CorsairLink.log from the past 2+ days. I get some "Checksum Failure" in the logfile, but I have not gotten any error messages in the UI yet. I don't see any errors in SIngnalRGB either and it has been running with a constantly changing RGB effect the whole time.

CorsairLink.log

[EvanMulawski]

Thanks for the updates! I'm going to reduce the log level of the checksum error to Warning and track those differently as the device is still performing as expected.

[chjohans]

@EvanMulawski After SignalRGB auto-updated to V2.3.48-beta the error is back, it was solved by replacing the file Platinum_Pro_XT_Elite_AIO.js as per your post further up.

Will this change somehow be included with SignalRGB at some point or do we have to replace this file on every update?

[EvanMulawski]

@EvanMulawski After SignalRGB auto-updated to V2.3.48-beta the error is back, it was solved by replacing the file Platinum_Pro_XT_Elite_AIO.js as per your post further up.

I believe SignalRGB re-downloads all plugins when updating.

Will this change somehow be included with SignalRGB at some point or do we have to replace this file on every update?

The SignalRGB team will need to implement the changes their official version of the plugin.

/cc @Heal-Bot

[chjohans]

@EvanMulawski After SignalRGB auto-updated to V2.3.48-beta the error is back, it was solved by replacing the file Platinum_Pro_XT_Elite_AIO.js as per your post further up.

That I also figured out since the file had changed with the update, and re-placing it again solved the issue.

I believe SignalRGB re-downloads all plugins when updating.

Will this change somehow be included with SignalRGB at some point or do we have to replace this file on every update?

The SignalRGB team will need to implement the changes their official version of the plugin.

I also figured out that much, I guess I should have been crystal clear in my question. What I really wonder is where the changed file comes from, is it something you just did as a quick-fix to get this to work, or has it been done by the SignalRGB team, or both? And are someone communicating these needed changes to their dev team?

I am guessing that this (bug?) in the Platinum_Pro_XT_Elite_AIO.js also affect other apps (besides FanControl) ability to run in parallel with SIgnalRGB even when they have implemented the shared mutex correctly?

/cc @Heal-Bot