search

MatKlucznyk / Qsys

Qsys library for S#P and SIMPL
MIT License
53 stars 18 forks source link

If I applied redundancy in QSYS, how could I use these modules for control redundancy? #6

Closed winghoing closed 1 year ago

MatKlucznyk commented 2 years ago

Right now you would have to duplicate your controls and add a second core module. Proper redundancy failover is on the roadmap.

swmarks commented 2 years ago

@MatKlucznyk In the meantime while you're working on redundant core support, would you recommend using the backup core handler module from the official Crestron modules to determine when the backup core becomes active? It seems like you'd want to buffer the SIMPL program's inputs/outputs to/from one set of modules and have the backup handler basically act as a toggle that activates the appropriate set of buffers for each module. My big remaining question is what would the inactive set of modules be doing while their core is in standby? Does a core in standby send real time feedback updates and/or would there be any issues/delays/manual intervention required with the backup modules' feedback syncing up when the backup core goes active?

winghoing commented 2 years ago

how could I apply redundancy using the latest module v2.3.0? I need to control the active backup core if the main core is inactive. Then, I need to control the main core again if the main core state change to active again.

MatKlucznyk commented 2 years ago

As stated above, you will need to duplicate your controls and manage it with the IsConnected feedback.

MatKlucznyk commented 2 years ago

Since I do not have two cores I can test this with, can someone explain how the cores behave, specifically regarding network connectivity/QRC access, when the primary core experiences a fault? Does the primary core close current QRC connections when the secondary core becomes active? Does the secondary core refuse connections when it is not active?

hunlock commented 2 years ago

The inactive core will not close QRC connection, but it will reply as 'inactive' when engine status is queried, any active change groups are gone and commands sent to it won't have any real action. Your module does poll for active status, but doesn't act on it.

I was going to try and use this to hop to the active core, but I realized there's no disposal being done and I ended up with a whole mess.

I think you need to know the following: IP/host of coreOne and CoreTwo If address exists on coreTwo open and maintain connection to both.

Poll for active on both

Build change group for active core

If active changes, dispose all, rebuild change group for new active.

It would also be beneficial for a manual dispose/rebuild to be exposed for manual rebuild.

On Sun, Sep 25, 2022 at 09:40 Mat Klucznyk @.***> wrote:

Since I do not have two cores I can test this with, can someone explain how the cores behave, specifically regarding network connectivity/QRC access, when the primary core experiences a fault? Does the primary core close current QRC connections when the secondary core becomes active? Does the secondary core refuse connections when it is not active?

— Reply to this email directly, view it on GitHub https://github.com/MatKlucznyk/Qsys/issues/6#issuecomment-1257196766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZU2RBNXH53QCTJ2TBC6GQDWABI5JANCNFSM5LEXTLXA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- @.*** +1.517.304.9728

MatKlucznyk commented 2 years ago

The inactive core will not close QRC connection, but it will reply as 'inactive' when engine status is queried, any active change groups are gone and commands sent to it won't have any real action. Your module does poll for active status, but doesn't act on it. I was going to try and use this to hop to the active core, but I realized there's no disposal being done and I ended up with a whole mess. I think you need to know the following: IP/host of coreOne and CoreTwo If address exists on coreTwo open and maintain connection to both. Poll for active on both Build change group for active core If active changes, dispose all, rebuild change group for new active. It would also be beneficial for a manual dispose/rebuild to be exposed for manual rebuild. On Sun, Sep 25, 2022 at 09:40 Mat Klucznyk @.> wrote: Since I do not have two cores I can test this with, can someone explain how the cores behave, specifically regarding network connectivity/QRC access, when the primary core experiences a fault? Does the primary core close current QRC connections when the secondary core becomes active? Does the secondary core refuse connections when it is not active? — Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZU2RBNXH53QCTJ2TBC6GQDWABI5JANCNFSM5LEXTLXA . You are receiving this because you are subscribed to this thread.Message ID: @.> -- @.*** +1.517.304.9728

Got it, should be easy enough then. Thank you

JD-Harding commented 1 year ago

Hi Matt, Seem to have an issue where MuteToggle on the Fader module can get out of sync with the Core which effectively locks that control. To resolve you need to action the button in Qsys designer to bring it back in sync and then it will continue working as per normal. The only time I've been able to recreate this is if you toggle a button rapidly for a few seconds, but we have had anecdotal reports that it does just happen during normal usage.

Is this something you've seen before / are aware of?

MatKlucznyk commented 1 year ago

Update the SIMPL modules and .clz to the latest code from the SIMPL folder. The mute toggle is no longer dependent on subscription responses. All controls now read the command response as well as the subscription update. Even if for some reason the control goes out of sync it will now update properly.

JD-Harding commented 1 year ago

Thanks Mat - I'll give it a go!

MatKlucznyk commented 1 year ago

The latest pull request merge added support for core redundancy. A release will be coming shortly.

hunlock commented 1 year ago

Will this support destroying old instances and building new ones based on the current master meaning it should run for ever, or will it only support a few flips before a program reset?

On Fri, Oct 6, 2023 at 1:47 PM Mat Klucznyk @.***> wrote:

The latest pull request merge added support for core redundancy. A release will be coming shortly.

— Reply to this email directly, view it on GitHub https://github.com/MatKlucznyk/Qsys/issues/6#issuecomment-1751182161, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZU2RBIOFIA33L2EOFZJTQ3X6A73ZAVCNFSM5LEXTLXKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZVGEYTQMRRGYYQ . You are receiving this because you commented.Message ID: @.***>

MatKlucznyk commented 1 year ago

This supports re-initialization on whichever core signals it is the active core and will reconnect if both cores go offline and return online.

From my testing, I was able to reboot two cores back and forth without issue.

hunlock commented 1 year ago

Awesome, thanks for the update!

On Fri, Oct 6, 2023 at 4:28 PM Mat Klucznyk @.***> wrote:

This supports re-initialization on whichever core signals it is the active core and will reconnect if both cores go offline and return online.

From my testing, I was able to reboot two cores back and forth without issue.

— Reply to this email directly, view it on GitHub https://github.com/MatKlucznyk/Qsys/issues/6#issuecomment-1751368279, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZU2RBJLWPWCYX2DHQQNFO3X6BSXFAVCNFSM5LEXTLXKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZVGEZTMOBSG44Q . You are receiving this because you commented.Message ID: @.***>