search

jeff-winn / homebridge-automower-platform

A platform plugin for Homebridge to connect your Husqvarna Automower and Gardena Smart robotic lawn mowers to Apple HomeKit.
MIT License
10 stars 0 forks source link

Limits being exceeded breaking startup #195

Closed jeff-winn closed 1 year ago

jeff-winn commented 1 year ago

Describe The Bug: I’ve seen a noticeable increase in the number of 429s occurring on startup of the plug-in. Given there are only two calls that occur, and they’re using different service APIs I’m under the impression that the single call per second now applies across the Husqvarna service endpoints.

This 429 in between the services allow authentication but fail startup as the call to discover the mowers fails to complete.

To Reproduce: It really depends on timing and responses if the APIs will respond quickly enough.

Expected behavior: The 429 should not occur and startup should complete normally.

Logs:

[11/22/2022, 9:45:47 PM] [Developer #1] Initializing Homebridge Automower Platform platform...
[11/22/2022, 9:45:47 PM] [Developer #1] Discovering new mowers...
[11/22/2022, 9:45:47 PM] [Developer #1] Logging into the Husqvarna platform...
[11/22/2022, 9:45:48 PM] [Developer #1] Sending request [1/5]: 77b3ebbc-142a-4fc7-ac06-c6b50f4bdb10
 {"url":"https://api.authentication.husqvarnagroup.dev/v1/oauth2/token","method":"POST","headers":{"content-type":["application/x-www-form-urlencoded;charset=UTF-8"],"x-application-id":["homebridge-automower-platform"]},"body":"grant_type=client_credentials&client_id=REDACTED&client_secret=REDACTED&scope=iam%3Aread%20amc%3Aapi"}
[11/22/2022, 9:45:48 PM] [Developer #1] Received response: 77b3ebbc-142a-4fc7-ac06-c6b50f4bdb10
 {"status":200,"statusText":"OK","headers":{"access-control-allow-headers":["Origin,X-Requested-With,X-Internal-Request-Id,X-External-Request-Id,Content-Type,Accept,X-IAM-Session,Authorization,Authorization-Provider"],"access-control-allow-methods":["GET, POST, OPTIONS, PUT, PATCH, DELETE"],"access-control-allow-origin":["*"],"cache-control":["no-store, no-cache, must-revalidate, proxy-revalidate"],"connection":["close"],"content-length":["1251"],"content-type":["application/json; charset=utf-8"],"date":["Wed, 23 Nov 2022 02:45:48 GMT"],"etag":["W/\"4e3-aJPGVJsLamGPHgj8cH0Bc5ytcgY\""],"expect-ct":["max-age=0"],"expires":["0"],"pragma":["no-cache"],"referrer-policy":["no-referrer"],"strict-transport-security":["max-age=15552000; includeSubDomains"],"surrogate-control":["no-store"],"via":["1.1 2bb4560ed08e22695c5989511be7b43e.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCJmcHp2joEFeHA="],"x-amz-cf-id":["PSvK7mjgdap09wwHZ5pJ3I-y7cMep3-f5szyzSQ0mW0K-DhTbpNofw=="],"x-amz-cf-pop":["ORD58-P3"],"x-amzn-remapped-connection":["keep-alive"],"x-amzn-remapped-content-length":["1251"],"x-amzn-remapped-date":["Wed, 23 Nov 2022 02:45:48 GMT"],"x-amzn-requestid":["74c5d5a1-0848-473c-be8e-a7b0e5e137af"],"x-cache":["Miss from cloudfront"],"x-content-type-options":["nosniff"],"x-dns-prefetch-control":["off"],"x-frame-options":["SAMEORIGIN"],"x-internal-request-id":["778125824"],"x-permitted-cross-domain-policies":["none"],"x-xss-protection":["0"]},"body":{"access_token":"REDACTED","scope":"iam:read amc:api","expires_in":86399,"provider":"husqvarna","user_id":"REDACTED","token_type":"Bearer"}}
[11/22/2022, 9:45:48 PM] [Developer #1] Logged in!
[11/22/2022, 9:45:48 PM] [Developer #1] Sending request [1/5]: 7700181d-9182-4ee7-8d04-0da306d170be
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:45:48 PM] [Developer #1] Received response: 7700181d-9182-4ee7-8d04-0da306d170be
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:45:48 GMT"],"via":["1.1 26c731836eb716e46fe9852a7aaeb508.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCJmhHJoDoEFgLw="],"x-amz-cf-id":["nDwNRvaREsdVxgdvxmqtnWHEVtz-kUmBGAiSScJiB17OvWEDDseoGA=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["89afc056-4f30-47fa-8d94-2b9372205b14"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:45:50 PM] [Developer #1] Sending request [2/5]: 7700181d-9182-4ee7-8d04-0da306d170be
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:45:50 PM] [Developer #1] Received response: 7700181d-9182-4ee7-8d04-0da306d170be
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:45:50 GMT"],"via":["1.1 999cbfbbbe9ca154a9a49a64d965791e.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCJmyEvTjoEFnWw="],"x-amz-cf-id":["6HztP1th_8m1vB5rhp6HP5nY-v2Y-oFC_FevdnUWjcyGpuIxKtXlEg=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["27718efb-45e7-4754-85a6-0dc37e912cd0"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:45:51 PM] [Developer #1] Sending request [3/5]: 7700181d-9182-4ee7-8d04-0da306d170be
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:45:52 PM] [Developer #1] Received response: 7700181d-9182-4ee7-8d04-0da306d170be
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:45:52 GMT"],"via":["1.1 63a4e61b4b406c1536ed36dc35d5a134.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCJnBGVKDoEFjYw="],"x-amz-cf-id":["ruV0vc0EPEbAlKtWBmBbvIOq1gjX6N8pCEZv7iVBqE4V7foFhELaFA=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["94692498-0179-49a5-ac2f-c1325dbca1b8"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:45:53 PM] [Developer #1] Sending request [4/5]: 7700181d-9182-4ee7-8d04-0da306d170be
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:45:54 PM] [Developer #1] Received response: 7700181d-9182-4ee7-8d04-0da306d170be
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:45:53 GMT"],"via":["1.1 a1a6e8c498ac24bc4342d4acc68709cc.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCJnVHoPDoEF6iA="],"x-amz-cf-id":["NSVxKpLLW27H8CY8sYmqQMugVdyEnX9tF9XJK-r1gJ54LBtn5ezw5g=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["0195f6b4-2734-42e6-abc4-ca299351f06b"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:45:55 PM] [Developer #1] Sending request [5/5]: 7700181d-9182-4ee7-8d04-0da306d170be
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:45:56 PM] [Developer #1] Received response: 7700181d-9182-4ee7-8d04-0da306d170be
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:45:55 GMT"],"via":["1.1 a4183ed156210b7ee2a8130a2e031ad4.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCJnpH44DoEF7Xg="],"x-amz-cf-id":["4UFDT13Rv5X7ZqiznMks6R8ZEKAEPcApDvHzXrwmaQbMvlw8-wqw9w=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["efc32410-830c-4286-9133-d80e8b792b6f"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:45:58 PM] [Developer #1] An unexpected error occurred while starting the plugin. UnexpectedServerError: ERR: [429] {"message":"Limit Exceeded"}
    at DefaultErrorFactory.unexpectedServerError (C:\Users\jeffw\Code\homebridge-automower-platform\src\errors\errorFactory.ts:102:16)
    at AutomowerClientImpl.<anonymous> (C:\Users\jeffw\Code\homebridge-automower-platform\src\clients\automowerClient.ts:177:37)
    at Generator.next (<anonymous>)
    at fulfilled (C:\Users\jeffw\Code\homebridge-automower-platform\dist\clients\automowerClient.js:5:58) {
  errorCode: 'ERR0000'
}

Plugin Config:

{
    "name": "Developer #1",
    "lang": "en",
    "appKey": "REDACTED",
    "authentication_mode": "client_credentials",
    "application_secret": "REDACTED",
    "platform": "Homebridge Automower Platform"
}

Environment:

jeff-winn commented 1 year ago

This likely just needs artificial latency added (via a thread sleep) to the client code to keep the limits from being exceeded.

The issue is then that which time is the rule being applied?

The answer to that will determine when the timer should start. It’ll just need to balance operational consistency with performance.

jeff-winn commented 1 year ago

This seems to be very sporadic when Husqvarna signals too many requests. Even intentionally forcing two requests to occur at the same time I cannot get this to reproduce.

jeff-winn commented 1 year ago

I managed to have this issue pop up again tonight, this time gathering logs when it showed up. (They will be attached later)

The 429s were consistently appearing on the 2nd step during startup to get the list of mowers. Auth always works, the step to get the mowers seems to fail at peak times due to the 429 limits being exceeded.

This time, I threw a 1 second then a 60 second sleep in between the two calls and neither resolved the issue. I can only assume the problem is with Husqvarna and the AWS endpoint blocking traffic due to volume.

jeff-winn commented 1 year ago

This request contained a forced delay of 1 second in between the login and get mowers web service calls:

[11/22/2022, 9:52:14 PM] [Developer #1] Initializing Homebridge Automower Platform platform...
[11/22/2022, 9:52:14 PM] [Developer #1] Discovering new mowers...
[11/22/2022, 9:52:14 PM] [Developer #1] Logging into the Husqvarna platform...
[11/22/2022, 9:52:14 PM] [Developer #1] Sending request [1/5]: f8add521-4537-4370-9806-9f99dad87452
 {"url":"https://api.authentication.husqvarnagroup.dev/v1/oauth2/token","method":"POST","headers":{"content-type":["application/x-www-form-urlencoded;charset=UTF-8"],"x-application-id":["homebridge-automower-platform"]},"body":"grant_type=client_credentials&client_id=REDACTED&client_secret=REDACTED&scope=iam%3Aread%20amc%3Aapi"}
[11/22/2022, 9:52:15 PM] [Developer #1] Received response: f8add521-4537-4370-9806-9f99dad87452
 {"status":200,"statusText":"OK","headers":{"access-control-allow-headers":["Origin,X-Requested-With,X-Internal-Request-Id,X-External-Request-Id,Content-Type,Accept,X-IAM-Session,Authorization,Authorization-Provider"],"access-control-allow-methods":["GET, POST, OPTIONS, PUT, PATCH, DELETE"],"access-control-allow-origin":["*"],"cache-control":["no-store, no-cache, must-revalidate, proxy-revalidate"],"connection":["close"],"content-length":["1251"],"content-type":["application/json; charset=utf-8"],"date":["Wed, 23 Nov 2022 02:52:15 GMT"],"etag":["W/\"4e3-uu/Oa6+mYIqKSjfP+BfXWCoAYdk\""],"expect-ct":["max-age=0"],"expires":["0"],"pragma":["no-cache"],"referrer-policy":["no-referrer"],"strict-transport-security":["max-age=15552000; includeSubDomains"],"surrogate-control":["no-store"],"via":["1.1 1385d7982831de93d1706949eb3a9f82.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCKi6FpsDoEFcSA="],"x-amz-cf-id":["binzFe938NwuD6tkRaSigJCM4xnPKELOeMnzFltmQ_BVbd5RFiGrUA=="],"x-amz-cf-pop":["ORD58-P3"],"x-amzn-remapped-connection":["keep-alive"],"x-amzn-remapped-content-length":["1251"],"x-amzn-remapped-date":["Wed, 23 Nov 2022 02:52:15 GMT"],"x-amzn-requestid":["6da26c7a-cb9a-4d05-811c-f5f171597b1e"],"x-cache":["Miss from cloudfront"],"x-content-type-options":["nosniff"],"x-dns-prefetch-control":["off"],"x-frame-options":["SAMEORIGIN"],"x-internal-request-id":["311171057"],"x-permitted-cross-domain-policies":["none"],"x-xss-protection":["0"]},"body":{"access_token":"REDACTED","scope":"iam:read amc:api","expires_in":86399,"provider":"husqvarna","user_id":"REDACTED","token_type":"Bearer"}}
[11/22/2022, 9:52:15 PM] [Developer #1] Logged in!
[11/22/2022, 9:52:15 PM] [Developer #1] Sending request [1/5]: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:52:15 PM] [Developer #1] Received response: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:52:15 GMT"],"via":["1.1 d2781cbc5b051eea1dbcba637ac10784.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCKi-HFOjoEF2QQ="],"x-amz-cf-id":["9YRFbpaO10cNgQ-myFlbfBwX3YK2G-MNoGgEsmfAkF3bOWGBnzrwrw=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["a85a05d6-40dd-476d-b105-c6e1a0fb04f8"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:52:17 PM] [Developer #1] Sending request [2/5]: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:52:17 PM] [Developer #1] Received response: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:52:17 GMT"],"via":["1.1 d2781cbc5b051eea1dbcba637ac10784.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCKjRHIKjoEF73A="],"x-amz-cf-id":["TzxzAXoijlgyevk6S8U0DthMKmrP9AZ3ubaSdnc-b7DMm6EgUBUqJw=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["ef38e3fa-c87a-47f0-87e0-dc443fc6f137"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:52:18 PM] [Developer #1] Sending request [3/5]: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:52:19 PM] [Developer #1] Received response: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:52:19 GMT"],"via":["1.1 74b6b6a4f766ff964b6f4249af5f5a90.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCKjhFJ6DoEFeAA="],"x-amz-cf-id":["qW8Qk3f5ZqEcwPQvVAirFzrazNn3VsuvYCD6Ch0nGVMRqNqlmoonRQ=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["77c96316-0e7e-41f3-8344-943af13a24fe"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:52:20 PM] [Developer #1] Sending request [4/5]: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:52:21 PM] [Developer #1] Received response: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:52:21 GMT"],"via":["1.1 28932a3f25cf6715295f68710a6431a4.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCKj2FcTjoEFf3w="],"x-amz-cf-id":["dfyJwaNfkkJ2-AhgKs9uUGEE2lJ_AmVv8x9zW6IeiMfbsr3cmEsvLw=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["b2796471-96ba-4852-92d7-8ce68b4d291d"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:52:22 PM] [Developer #1] Sending request [5/5]: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 9:52:23 PM] [Developer #1] Received response: 0d5ee2c8-ea9c-4027-b2dc-cb8aa375b629
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 02:52:22 GMT"],"via":["1.1 efcaf943b1bc2a100ddcb9442a62d000.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCKkHFWzjoEF9XA="],"x-amz-cf-id":["piYaQfapBFdy7jS9_TPxTP_5v_5WpdfZcJc4nFUqiWEkb22FsDyD6g=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["e44c2e08-9906-48d9-84ca-777029ad631c"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 9:52:24 PM] [Developer #1] An unexpected error occurred while starting the plugin. UnexpectedServerError: ERR: [429] {"message":"Limit Exceeded"}
    at DefaultErrorFactory.unexpectedServerError (C:\Users\jeffw\Code\homebridge-automower-platform\src\errors\errorFactory.ts:102:16)
    at AutomowerClientImpl.<anonymous> (C:\Users\jeffw\Code\homebridge-automower-platform\src\clients\automowerClient.ts:177:37)
    at Generator.next (<anonymous>)
    at fulfilled (C:\Users\jeffw\Code\homebridge-automower-platform\dist\clients\automowerClient.js:5:58) {
  errorCode: 'ERR0000'
}
jeff-winn commented 1 year ago

This request contained a forced delay of 60 seconds in between the login and get mowers web service calls:

[11/22/2022, 10:00:03 PM] [Developer #1] Initializing Homebridge Automower Platform platform...
[11/22/2022, 10:00:03 PM] [Developer #1] Discovering new mowers...
[11/22/2022, 10:00:03 PM] [Developer #1] Logging into the Husqvarna platform...
[11/22/2022, 10:00:03 PM] [Developer #1] Sending request [1/5]: 7cb9c025-1916-4d3b-8ee9-7c154a0101de
 {"url":"https://api.authentication.husqvarnagroup.dev/v1/oauth2/token","method":"POST","headers":{"content-type":["application/x-www-form-urlencoded;charset=UTF-8"],"x-application-id":["homebridge-automower-platform"]},"body":"grant_type=client_credentials&client_id=REDACTED&client_secret=REDACTED&scope=iam%3Aread%20amc%3Aapi"}
[11/22/2022, 10:00:06 PM] [Developer #1] Received response: 7cb9c025-1916-4d3b-8ee9-7c154a0101de
 {"status":200,"statusText":"OK","headers":{"access-control-allow-headers":["Origin,X-Requested-With,X-Internal-Request-Id,X-External-Request-Id,Content-Type,Accept,X-IAM-Session,Authorization,Authorization-Provider"],"access-control-allow-methods":["GET, POST, OPTIONS, PUT, PATCH, DELETE"],"access-control-allow-origin":["*"],"cache-control":["no-store, no-cache, must-revalidate, proxy-revalidate"],"connection":["close"],"content-length":["1251"],"content-type":["application/json; charset=utf-8"],"date":["Wed, 23 Nov 2022 03:00:06 GMT"],"etag":["W/\"4e3-D3PrkZqkrLU01tN5W3xDabbHqCo\""],"expect-ct":["max-age=0"],"expires":["0"],"pragma":["no-cache"],"referrer-policy":["no-referrer"],"strict-transport-security":["max-age=15552000; includeSubDomains"],"surrogate-control":["no-store"],"via":["1.1 62bd0bb5da0f14d62bdae0c2ab5e672e.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCLsJHWdjoEFbMw="],"x-amz-cf-id":["7EDjCTPtNusQ5TkgNSfoUfTehYsYtrXQVA6_H51oK1mA1tv8Sgw37Q=="],"x-amz-cf-pop":["ORD58-P3"],"x-amzn-remapped-connection":["keep-alive"],"x-amzn-remapped-content-length":["1251"],"x-amzn-remapped-date":["Wed, 23 Nov 2022 03:00:06 GMT"],"x-amzn-requestid":["24fb61e7-d02c-4974-a185-3f3346c7dbdc"],"x-cache":["Miss from cloudfront"],"x-content-type-options":["nosniff"],"x-dns-prefetch-control":["off"],"x-frame-options":["SAMEORIGIN"],"x-internal-request-id":["255484714"],"x-permitted-cross-domain-policies":["none"],"x-xss-protection":["0"]},"body":{"access_token":"REDACTED","scope":"iam:read amc:api","expires_in":86398,"provider":"husqvarna","user_id":"REDACTED","token_type":"Bearer"}}
[11/22/2022, 10:00:06 PM] [Developer #1] Logged in!
[11/22/2022, 10:01:06 PM] [Developer #1] Sending request [1/5]: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 10:01:07 PM] [Developer #1] Received response: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 03:01:07 GMT"],"via":["1.1 999cbfbbbe9ca154a9a49a64d965791e.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCL2BFEMjoEFSwQ="],"x-amz-cf-id":["OKQTxttn9FfWgX4B7tuMvPoqk-W-FRavNI_qdaJUqYRMZOhuLkU8ow=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["f5736daa-57b9-465d-ba19-b3e7bcc6706f"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 10:01:08 PM] [Developer #1] Sending request [2/5]: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 10:01:08 PM] [Developer #1] Received response: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 03:01:08 GMT"],"via":["1.1 63a4e61b4b406c1536ed36dc35d5a134.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCL2PEkqjoEFzQw="],"x-amz-cf-id":["ray0AP6W2KS_JD_KCpcw_MYjRCLuCcfAaaJIbaI_qfHK2xwbI4puEw=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["a85e5569-817b-490e-aa79-eec5c7b8fe29"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 10:01:10 PM] [Developer #1] Sending request [3/5]: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 10:01:10 PM] [Developer #1] Received response: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 03:01:10 GMT"],"via":["1.1 90bcfd5a3df2cb2e3f60dc3995832c76.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCL2mEieDoEF2bA="],"x-amz-cf-id":["2-829sdFmfku8D2vpHHWmqQV4lWUyVT6plD0zCj5TPvfQMLsBICCVQ=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["ea237d89-7f5e-4295-9fc1-aa723e412eb7"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 10:01:12 PM] [Developer #1] Sending request [4/5]: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 10:01:12 PM] [Developer #1] Received response: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 03:01:12 GMT"],"via":["1.1 0bff98411be7553fe46a16d779ea3486.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCL24FWIDoEFhbQ="],"x-amz-cf-id":["VgPDMHn__aADRnAdZ2EL5ovdzrs3NFoJ_7bbcTdCrlwbciHSfpGr_Q=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["c226c712-cdef-40b9-8089-fc88ecd03d1c"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 10:01:14 PM] [Developer #1] Sending request [5/5]: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"url":"https://api.amc.husqvarna.dev/v1/mowers","method":"GET","headers":{"authorization":["REDACTED"],"authorization-provider":["husqvarna"],"x-api-key":["REDACTED"],"x-application-id":["homebridge-automower-platform"]}}
[11/22/2022, 10:01:14 PM] [Developer #1] Received response: 6b6e9a1a-067d-4167-8434-55f5da6a74a2
 {"status":429,"statusText":"Too Many Requests","headers":{"connection":["close"],"content-length":["28"],"content-type":["application/json"],"date":["Wed, 23 Nov 2022 03:01:14 GMT"],"via":["1.1 4b8be55299e599fba7c9de026a1b5eca.cloudfront.net (CloudFront)"],"x-amz-apigw-id":["cCL3JEw9DoEFtAQ="],"x-amz-cf-id":["qyrLjyVVckpltvZ-v9cXaVkVkTXRlcwSkmE01xKHcsK6gbipTr0--w=="],"x-amz-cf-pop":["ORD58-P7"],"x-amzn-errortype":["LimitExceededException"],"x-amzn-requestid":["d53d153b-2074-4d90-a582-9e296a9e6ddd"],"x-cache":["Error from cloudfront"]},"body":{"message":"Limit Exceeded"}}
[11/22/2022, 10:01:16 PM] [Developer #1] An unexpected error occurred while starting the plugin. UnexpectedServerError: ERR: [429] {"message":"Limit Exceeded"}
    at DefaultErrorFactory.unexpectedServerError (C:\Users\jeffw\Code\homebridge-automower-platform\src\errors\errorFactory.ts:102:16)
    at AutomowerClientImpl.<anonymous> (C:\Users\jeffw\Code\homebridge-automower-platform\src\clients\automowerClient.ts:177:37)
    at Generator.next (<anonymous>)
    at fulfilled (C:\Users\jeffw\Code\homebridge-automower-platform\dist\clients\automowerClient.js:5:58) {
  errorCode: 'ERR0000'
}
jeff-winn commented 1 year ago

I sent an email over to the Husqvarna support team on 11/23/2022 @ 5:57 PM EST to ask for engagement to see if I can confirm my theory as to what's causing the rate limiting seen by the plugin.

My current prevailing theory is noisy neighbors causing the platform to rate limit as a whole, and not something the plugin may be doing in relation to this startup issue.

jeff-winn commented 1 year ago

I got the following response back from the Husqvarna support staff:

We have found an issue in how the quota is calculated in the system. A fix will be deployed on Monday. For now, we have increased the quota per key so there should be less requests with error 429.

Still plan to do a rate limiting client in the app anyway, but at least the issue should be resolved. This won't correct the issue with multiple requests being sent from Apple HomeKit in the case of scenes setting multiple mowers at once.

jeff-winn commented 1 year ago

Closing as resolved with Husqvarna indicating they'll be making a change on their end along with the rate limited client within the plugin.