ConvertLegacyData worked - how do we get rid of legacy trigger?

[rmasonjr]

Environment

HomeSeer

OS

[Windows]

HS Version

[v4.2.14.0)

Development

PSDK Version

[v1.4.2.0]

Language

[VB]

IDE

[VS2017]

Dev OS

[Windows]

Plugin

What plugin is this issue for? OMNI

Problem

How to get rid of legacy trigger and action data?

Description

I finally got ConvertLegacyData called and have successfully deserialized my legacy trigger and converted it to the new AbstractTriggerType. The problem is that it still gets called every time since there is no real way to get rid of the legacy trigger data.

How do we get rid of the inData after converting our triggers (and actions)?

Expected Behavior

no more inData

Steps to Reproduce

Provide steps so the HomeSeer team can reproduce the reported problem:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Screenshots

see attached screenshot

Logs

If applicable, include log output from the plugin console and/or the HomeSeer logs. If the logs take up more than a few lines, consider attaching a file.

[rmasonjr]

@jldubz or @spudwebb any ideas on how to get rid of the legacyTrigger after ConvertLegacyData is called?

[jldubz]

@rmasonjr ,

You have to resave the data manually or when a user updates the event trigger. Until this happens, there is no call to write the event data to the HS database so HS does not know it needs to resave it. We do this using a setting in the Z-Wave plugin. On startup, if the setting is set, the plugin pulls all of the triggers and actions that it owns, updates them, and writes them back. Future loading of the triggers/actions will no longer attempt to convert legacy data.

Call UpdatePlugTrigger(String, Int32, TrigActInfo) to manually update the trigger. Data will also be saved after a call to OnConfigItemUpdate() when true is returned. Check out the lifecycle for triggers.

[rmasonjr]

ok - this is my new ConvertLegacyData code, with the call to UpdatePlugTrigger and it's still not working.
ConvertLegacyData still gets called a crazy number of times which would indicate the legacy trigger data is still there:

Protected Overrides Function ConvertLegacyData(inData As Byte()) As Byte()
        Log(TriggerName & " ConvertLegacyData from trigger was called...", LogType.LOG_TYPE_DEBUG)
        Dim tai As TrigActInfo
        Dim legacyTrigger As UserCodeTrigger
        If inData IsNot Nothing AndAlso inData.Length > 0 Then
            Try
                'attempt to deserialize the byte array to your custom class
                legacyTrigger = TrigActInfo.DeserializeLegacyData(Of UserCodeTrigger)(inData)
            Catch ex As Exception
                'if there was a problem - set the legacy data object to null
                legacyTrigger = Nothing
            End Try
        End If

        If legacyTrigger Is Nothing Then
            Return New Byte() {}
        End If

        'Replace the following line with your own code to upgrade your legacy data to modern config page
        ConfigPage = ConvertLegacyDataToConfigPage(legacyTrigger)

        ''I feel like there should be something here to delete the legacy data since it is now converted (inData?)
        tai = New TrigActInfo()
        tai.DataIn = Nothing
        Log("---> Clearing old trigger data... EventRef: " & EventRef, LogType.LOG_TYPE_DEBUG)
        hs4.UpdatePlugTrigger(OMNI_NAME, EventRef, tai)

        'return the Data property to automatically convert the ConfigPage to a byte array for later storage
        Return Data
    End Function

[jldubz]

@rmasonjr ,

You should not call to update a trigger during any of the lifecycle methods for a trigger type. This will produce unexpected results because the "old" data is actively being handled by HS and may overwrite any changes you make. There are very specific moments when the data is saved during the life of a trigger instance. See the lifecycle diagram in the AbstractTriggerType class reference.

UpdatePlugTrigger() should only be called during startup or from a dedicated process thread.

[rmasonjr]

I see the lifecycle of a trigger diagram for New and Edit. I'm trying to figure out how to get rid of the legacy trigger info so that ConvertLegacyData doesnt get called a crazy number of times.

[jldubz]

@rmasonjr

We do this using a setting in the Z-Wave plugin. On startup, if the setting is set, the plugin pulls all of the triggers and actions that it owns, updates them, and writes them back. Future loading of the triggers/actions will no longer attempt to convert legacy data.

UpdatePlugTrigger() should only be called during startup or from a dedicated process thread.

[rmasonjr]

@jldubz Ok - I am no longer using ConvertLegacyData, and I wrote a function to get all the triggers owned by my plugin and it will be called on startup:

Public Function ConvertTriggers() As Boolean
        Dim legacyTrigger As UserCodeTrigger
        Dim newTrigger As ArmDisarmTriggerType
        Dim trigActs As TrigActInfo()

        Try
            trigActs = hs4.GetTriggersByInterface(OMNI_NAME)

            For Each trigAct In trigActs
                Try
                    legacyTrigger = TrigActInfo.DeserializeLegacyData(Of UserCodeTrigger)(trigAct.DataIn)
                    newTrigger = New ArmDisarmTriggerType

                    'what do I do here?

                    hs4.UpdatePlugTrigger(OMNI_NAME, trigAct.evRef, trigAct)

                Catch ex As Exception
                    ''could not deserialize to my legacy UserCodeTrigger
                End Try
            Next

        Catch ex As Exception
            Log("ConvertTriggers: " & ex.Message, LogType.LOG_TYPE_DEBUG)
        End Try

    Return True
End Function

Can you fill in the blanks on what I need to do to update and write them back? UpdatePlugTrigger wants a TrigActInfo - I thought we were using the new AbstractTriggerType?

I think I'm close I just have no reference on how to update a trigger and write it back.

[rmasonjr]

@jldubz can you give me some pointers on the above post?

[jldubz]

@rmasonjr , thanks for the bump. Apologies for the delay. Here is a sample from the Z-Wave plugin routine you should be able to reuse.

'Scans all actions and converts the data stored in their configuration to JUI - this is run on startup when a flag is set to prevent it from running multiple times and doing unnecessary work.
'Do the same thing for triggers while using HsSystem.GetTriggersByInterface() and HsSystem.UpdatePlugTrigger() respectively
Private Sub CheckEventActionIntegrity()
    LogInfo("Scanning Event Actions to verify that legacy data was loaded successfully")
    Dim pluginActions = HsSystem.GetActionsByInterface(IFACE_NAME)
    If pluginActions.Count < 1
        Exit Sub
    End If

    Dim actionTypeInstance As ZwaveActionType
    Dim newTrigActInfo As TrigActInfo
    For Each pluginAction In pluginActions
        If pluginAction.TANumber <> 1
            Console.WriteLine($"Action {pluginAction.UID} is not a regular ZWave action")
            Continue For
        End If
        Try
            'Create a new action instance with existing data
            actionTypeInstance = New ZwaveActionType(pluginAction.UID, pluginAction.SubTANumber, pluginAction.evRef, pluginAction.DataIn, Nothing)
            'Trigger validation - this executes ZwaveActionType.GenConfigPageForLegacyEvent()
            If Not actionTypeInstance.IsFullyConfigured()
                LogWarning($"Event: {pluginAction.evRef} Action: {pluginAction.UID} is not fully configured and may not have loaded correctly")
            Else 
                newTrigActInfo = New TrigActInfo()
                newTrigActInfo.evRef = pluginAction.evRef
                newTrigActInfo.UID = pluginAction.UID
                newTrigActInfo.TANumber = pluginAction.TANumber
                newTrigActInfo.SubTANumber = pluginAction.SubTANumber
                newTrigActInfo.Instance = pluginAction.Instance
                newTrigActInfo.DataIn = actionTypeInstance.Data
                Continue For
                'This will resave the action with updated info. 
                'It will not change the interfacename used to indicate what plugin owns it.
                HsSystem.UpdatePlugAction(IFACE_NAME, newTrigActInfo.evRef, newTrigActInfo)
            End If
        Catch exception As Exception
            LogError($"Event: {pluginAction.evRef} Action: {pluginAction.UID} Error: cannot be processed and may be corrupted - {exception.Message}")
            LogError(exception.StackTrace)
        End Try
    Next
    LogInfo("Done Scanning Event Actions")
End Sub

'From ZwaveActionType
' Executed when IsFullyConfigured() is called to convert existing data to a new format
Friend Sub GenConfigPageForLegacyEvent(legAct As EvACT_ZWAVE)
    ConfigPage = InitNewConfigPage().Page
    'Check if the network is selectable or if there is only one network available
    If ConfigPage.ContainsViewWithId(NetworkSelectListId)
        'If there are multiple networks to choose from
        ' determine if a network has been chosen
        If legAct.ZA_HomeIDRaw IsNot Nothing AndAlso legAct.ZA_HomeIDRaw.Length = 4 Then
            'Get the HomeId from the RawHomeId stored in the legacy event
            Dim selectedHomeId = HomeIDRawToString(legAct.ZA_HomeIDRaw)
            'Get the network select list so we can determine if the selected network is in the current list
            Dim networkSelectList = ConfigPage.GetViewById(Of SelectListView)(NetworkSelectListId)
            If Not networkSelectList.OptionKeys.Contains(selectedHomeId)
                'No selected network was found, but a selection is needed.
                ' This could be because the selected network is no longer a viable option
                Exit Sub
            End If

            networkSelectList.Selection = networkSelectList.OptionKeys.IndexOf(selectedHomeId)
            ConfigPage.UpdateViewById(networkSelectList)
            If Not ConfigPage.ContainsViewWithId(ActionTypeSelectListId)
                ConfigPage.AddView(GetActionTypeSelectList(selectedHomeId))
            End If
        End If
    End If

    Dim actionSelectList = ConfigPage.GetViewById(Of SelectListView)(ActionTypeSelectListId)
    Dim selectedActionKey = Convert.ToInt32(legAct.ZA_subtype)
    If legAct.ZA_subtype < 1 And SelectedSubActionIndex <> -1
        selectedActionKey = SelectedSubActionIndex
    End If
    If Not actionSelectList.OptionKeys.Contains(selectedActionKey.ToString())
        ConfigPage.AddView(GetNewErrorLabel("This action type is not supported by the selected network"))
        Exit Sub
    End If

    actionSelectList.Selection = actionSelectList.OptionKeys.IndexOf(selectedActionKey.ToString())
    Dim selectedActionType As ZWave_EventAction_Type = [Enum].Parse(GetType(ZWave_EventAction_Type), selectedActionKey.ToString())
    StartConfigForType(selectedActionType)

    Select Case selectedActionType
        Case ZWave_EventAction_Type.ZW_Protection,
             ZWave_EventAction_Type.ZW_Suspend_Polling,
             ZWave_EventAction_Type.ZW_Resume_Polling,
             ZWave_EventAction_Type.ZW_Panic_Mode_On,
             ZWave_EventAction_Type.ZW_Panic_Mode_Off
            ConfigPage.AddView(GetNewErrorLabel("This action type is no longer supported"))

        Case ZWave_EventAction_Type.ZW_HSWX200_Actions
            GenHswx200PageFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_All_Off,
             ZWave_EventAction_Type.ZW_All_On
            GenControlAllFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_Scene_On,
             ZWave_EventAction_Type.ZW_Scene_Off
            GenSceneFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_PollNode
            GenPollNodeFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_Set_Configuration
            GenSetConfigFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_LED_On,
             ZWave_EventAction_Type.ZW_LED_Off
            GenLedFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_Set_User_Code
            GenSetUserCodeFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_Remove_User_Code
            GenRemoveUserCodeFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_Reset_Meter
            GenResetMeterFromLegacyAction(legAct)

        Case ZWave_EventAction_Type.ZW_Set_Meter_Rate
            GenSetMeterRateFromLegacyAction(legAct)

        Case Else
            Exit sub
    End Select

End Sub

[rmasonjr]

Thanks @jldubz - this helps so much!

[rmasonjr]

@jldubz I've been working with this and am so close, but still not 100% - a couple of questions:

Your code above never calls UpdatePlugAction - there is a "Continue For" right above - why is that?

Your GenConfigPageForLegacyEvent - you said it is called when IsFullyConfigured() is called - I tried calling mine in IsFullyConfigured(), but the legacy Action is not available. Where exactly are you calling it?

Finally, is there a way to just call something to save the new action?
For example, in my GenConfigPageForLegacyEvent

Friend Sub GenConfigPageForLegacyEvent(legAct As PanelClockAction)
        Log(ActionName & " GenConfigPageForLegacyEvent", LogType.LOG_TYPE_DEBUG)
        ConfigPage = InitNewConfigPage().Page

        If ConfigPage.ContainsViewWithId(CheckBoxId) Then
            If legAct IsNot Nothing Then
                Dim checkbox = ConfigPage.GetViewById(Of ToggleView)(CheckBoxId)
                checkbox.IsEnabled = True
                ConfigPage.UpdateViewById(checkbox)
            End If
        End If
    End Sub

What actually 'Saves' the new action? I know the lifecycle diagram shows where it is saved, but nothing I try seems to actually 'Save' the new action. Do I just call IsFullyConfigured() directly?

I'm so close, just cant find what actually 'Saves' the new action.

[spudwebb]

@rmasonjr , You definitely need to call UpdatePlugAction this is what actually saves the new action. I don't know why there was a "Continue For", but it shouldn't be there.

GenConfigPageForLegacyEvent is actually called from ConvertLegacyData like below. And ConvertLegacyData is called when you construct a new AbstractActionType and the deserializer fails to convert the data to the new format, in this case it assumes the data must be old format and need to be converted.

        protected override byte[] ConvertLegacyData(byte[] inData) {
            object legAct = null;
            if (inData != null && inData.Length > 10) {
                try {
                    legAct = new EvACT_ZWAVE(EventRef);

                    if (!HSPI_ZWave.Util.DeSerializeObject(ref inData, ref legAct, "[ActionBUI]", true))
                        legAct = null;
                }
                catch (Exception) {
                    legAct = null;
                }
            }

            if (legAct == null)
                return new byte[0];

            // Upgrade to modern config page
            GenConfigPageForLegacyEvent((EvACT_ZWAVE)legAct);
            return Data;
        }

Sorry for the confusion, the code posted by @jldubz is actually from an old version of the plugin and has been re-written in C# since, with a few changes.

So, to sum up, you must create a procedure which will be triggered at startup or through a setting. This procedure should

• scan all your event actions/triggers (GetActionsByInterface), and for each
• construct a new action/trigger in the new format from the old data (in our case it is ZwaveActionType which extends AbstractActionType)
• if conversion is successful, you save the action in the new format using UpdatePlugAction
• the actual conversion is done from ConvertLegacyData which will be called only when necessary.

[jldubz]

@rmasonjr

I forgot to remove the continue for from the code sample. This code is a little old. It shouldn't be there.

@spudwebb thanks for sharing the updated code in C#

[rmasonjr]

@jldubz @spudwebb

Success, guys! I am able to take Legacy Actions and successfully convert them to HS4 actions. All of the moving pieces are working great. Thanks so much for the help!

[jldubz]

@rmasonjr Awesome! Glad to hear. Thank you for your patience.