Handle inconsistencies in Schlage API responses for access code creation

[panzerchris]

Prerequisites

• Can this issue be reproduced in a sandbox workspace? No
• Does this issue represent a regression? Did it previously work? No

Description

We've identified an inconsistency in the Schlage API responses that affects our ability to provide accurate information about the status of access codes on Schlage locks. When creating an access code the Schlage API will occasionally return a 502 error such as:

POST /schlage-home/api/Devices/<device_id>/accesscodes

  "response": {
    "type": "https://api.allegion.com/errors/producers/SchlageHomeAPI/errorcodes/ResourceProviderBadGateway",
    "title": "The resource provider returned an invalid response.",
    "errors": {
      "Error Code: 34": [
        "Invalid access code schedule parameters."
      ]
    },

However, despite the error the access code will still appear when requesting all access codes set on the device:

GET /schlage-home/api/Devices/<device_id>/accesscodes

  "response": {
    "accessCodes": [ .... positive result here ... ]
  }

In some cases, as noted in internal discussions, we have received reports suggesting that the API returns an access code in the list of set codes, but the code may not be correctly set on the physical lock^discussion. Another manifestation of this problem is an access code disappearing within a short period of time.^discussion-2

Steps to Reproduce

We cannot reproduce this on demand. The Schlage API returns intermittent failures.

Expected behavior: We aim to provide reliable information about which access codes have been set, based on the responses from the Schlage API.

Actual behavior: We sometimes receive inconsistent responses from the Schlage API. It may claim an access code was set (by including it in the list of codes), even in cases where there's reason to believe the code might not be working on the physical lock.

Versions

Schlage API Integration Version: V1

Additional Information

n/a

Proposed Solutions

1. Implement more robust error handling and logging to catch and report these API inconsistencies.
2. Develop a system to automatically attempt the short-term mitigation strategy provided by Schlage:
   1. When an access code creation fails or appears inconsistent, automatically attempt a DELETE request for the access code using its ID.
   2. Immediately follow with a retry of the creation request.
   3. Log the results of this mitigation attempt for further analysis.

3. Create an internal monitoring system to track the frequency and patterns of these inconsistencies, which can help in discussions with Schlage and in developing our own long-term solutions.

   2684887333.log.txt

[codetheweb]

Related to a Slack conversation

[panzerchris]

Note, we have also experienced similar behaviour with Schlage when it returns a 504 error. The difference is when a 504 error occurs the access code usually disappears again within an hour. This causes a different inconsistency but there's an opportunity to address both issues here. See https://github.com/seamapi/public-tracker/issues/59

[razor-x]

@phpnode On call would like to know if you are working on this one this week.