search

csuermann / virtual-smart-home

Virtual Smart Home Skill for Amazon Alexa and Node-RED
27 stars 18 forks source link

Always deploys to eu-west-1? #1

Closed turbo2ltr closed 2 years ago

turbo2ltr commented 2 years ago

I'm in way over my head but I did get it deployed to AWS, I just couldn't find it in the AWS console because it ended up being deployed to EU-WEST-1 instead of the US-WEST-1

I had this environment variable set. AWS_DEFAULT_REGION=us-west-1

But I confess, I did accidentally have it set to EU prior to running serverless deploy successfully. I did run serverless deploy unsuccessfully with the variable set to EU, but I had a key typo keeping it from completing. But changing the env var didn't change anything, kept going to EU.

So I deleted the stack from AWS (which fails at first because it asks your permission to delete some bucket thingy). So at this point I have no stacks in any region.

I changed the serverless.yml to US and i made a copy of cloudFormation-eu-west-1.yml and renamed it because it started complaining about not being able to find cloudFormation-us-west-1.yml

I finally get it deployed to us-west-1 and when I try to add the ARN it complains it's not valid. I finally learn that alexa skills can't be deployed to us-west-1, only us-west-2. Made the updates again to serverless.yml and added a matching cloudformation file, and finally got it deployed where it's supposed to be.

After several other deployments everything seems connected. The node status says online in the flow. The device shows up in the connection node property window. Debug mode on and don't see any errors in the console. But just can't alexa to discover anything. I do see occasionally when I deploy the flows "PROACTIVE DISCOVERY FAILED", but it usually goes to online eventually.

I am using an AWS account that is not the same account as alexa. But on the initial connection setup it said I successfully paired the account. And eventually it started asking me to log in so I know I was in the correct account (the one with my alexa devices). I;ve tried searching for devices from the alexa app and the website countless times.. nothing.

Looking at CloudWatch, I do see this:

"payload": { "code": "SKILL_DISABLED_EXCEPTION", "description": "The user does not have a valid enablement for your skill." }

But I assure you it is enabled. I've even deleted it and put it back in multiple times, linking my account each time. So I can only think there is something crossed up with the accounts.

So account 1 has: All my echo devices, existing smart home devices, the dev account that I created the skill in, and the LWA security profile that I put in the skill's client ID/secret on the account linking page.

Account 2 is all the AWS / backend deployment.

I'm on hour 7 of this. all because I am trying to build my HA system out with zero dependencies outside my control. I've made alexa skills before and when I initially started this, I was thinking I was going to be able to run it on my own server. Didn't realize it required an AWS deployment. Sorry, I'm rambling.

csuermann commented 2 years ago

Hi @turbo2ltr,

Sorry to hear that deploying VSH to your own AWS account is such a hassle. I have to admit that the backend is not yet optimized for easy individual deployments. I open sourced the backend code mostly to allow users to inspect the code, not so much for spinning up their own environment. In any case, you seem to be on the finishing line, so congratulations ;-)

When it comes to account linking, it doesn't matter which Amazon account you use. It doesn't even need to be the one you use with your Alexa devices. The only thing that is absolutely essential is that you use the very same account for enabling the VSH skill in the Alexa app and for configuring the vsh-connection node in Node-RED.

My advice is to (re)-enable the VSH Alexa skill before configuring a new Node-RED vsh-connection.

For debugging, make sure to check the debug output in Node-RED, the IoT Core sections of the AWS console, the VSH table in DynamoDB as well as all the Cloudwatch Log Groups.

Hope that helps!

turbo2ltr commented 2 years ago

First, please don't be sorry. I understand the purpose of releasing the code and your instructions are very good for a very complicated setup. I'm just thick headed and refuse to give up. lol.

TL:DR: I got it working!

So now that I've made every mistake I ever could have, I started from scratch. I didn't reuse a single credential. Deleted all the IOT things, types, groups, stacks, and configs in rode-red. Created a new skill, new server deployment, new everything.

I found that I didn't make every mistake....I found that you must use the link provide in the instructions for the security profile. I had just deleted the previous profile and I just went to "security profiles" in the developer console and created a new profile. While creating it and adding the credentials to the skill went fine, it gave me what appeared to be a vague oauth related error when I tried to add the skill to alexa. I deleted that profile and created a new one using the link in the instructions and I could then add the skill in my alexa app. The only difference is your link asks for a privacy policy URL whereas the other method did not. My bad for not following the step exactly.

So at this point I had completed the 25 steps of the instructions, the server deployed, and skill added to my account. I then went to node-red and set up everything there. Linked it with the same account (even opened the linking URL in a private tab so it would force me to log into the correct account). Node says it's online.
I don't see any issue with the connection between node-red and the server. If I look at IoT Things, I see the connection and I see my plug node in the shadow document. I see MQTT messages in the console. If I send the node a powerState: ON, I see the publish message go out.

It just will not "talk" with the skill. The skill never finds any devices.

For whatever reason, it thinks the skill is disabled:

2021-07-31T17:01:07.756Z    3ba530bf-247b-46c1-bed0 ERROR   ✖ proactivelyDiscoverDevices FAILED! Request failed with status code 403. Response body: 
{
    "header": {
        "namespace": "System",
        "name": "Exception",
        "messageId": "da6a8649-e42e-4070-81b3"
    },
    "payload": {
        "code": "SKILL_DISABLED_EXCEPTION",
        "description": "The user does not have a valid enablement for your skill."
    }
}

SUCCESS!

Some code changes I made (rather blindly).

The endpoint for us-west-2 is not correct in helper.ts

function getEventGatewayUrl(region) {
  switch (region.toLowerCase()) {
    case 'eu-west-1':
      return 'https://api.eu.amazonalexa.com/v3/events'
    case 'us-east-1':
      return 'https://api.amazonalexa.com/v3/events'
    case 'us-west-2':
      return 'https://api.amazonalexa.com/v3/events'  //return 'https://api.fe.amazonalexa.com/v3/events'
  }
}

I believe this may be the one code issue that was causing the failure (aside from all my mistakes), but prior to that I did make one other change.

handlers/acceptGran.ts had a hardcoded eu-west-2 so I changed it as I wasn't sure where process.env.AWS_REGION was coming from. In other places in the code I saw process.env.VSH_IOT_REGION. Again I made these changes pretty much blindly.. I was desperate. lol.

await upsertTokens(
      {
        userId: event.profile.user_id,
        accessToken: tokenData.access_token,
        refreshToken: tokenData.refresh_token,
        email: event.profile.email,
        skillRegion: process.env.AWS_REGION || 'us-west-2'   // 'eu-west-2'
      },
      tokenData.expires_in
    )

Two minor points on the instructions:

Step 23, there is no need to add the endpoint to Europe/India. Assuming your AWS is set up in the region closest to you, since your house isn't going anywhere filling in the default is enough. If you use a US region, it actually won't let you add it to Europe/India. The North America endpoint only takes the US endpoint you aren't using. So I just added it to the default and nothing else.

In the Configuring Node-RED section step 4, you refer to step 19, but it's (now) step 20.

Thanks for all the work you've done on this project.

turbo2ltr commented 2 years ago

Looks like I jumped the gun a bit. The devices are indeed now discovered in the app but immediately show unresponsive in the app. They don't get updated and trying to control them does not produce any messages in the node-red cosole. This will have to wait for another day.

turbo2ltr commented 2 years ago

Seeing the following in /aws/lambda/virtual-smart-home-dev-metrics: 

2021-07-31T19:45:12.417Z    b9b8abcc-8351-442b-994f-1a5ed03d39a2    ERROR   Invoke Error    
{
    "errorType": "ResourceNotFoundException",
    "errorMessage": "Index AWS_Things does not exist. Please enable index by calling UpdateIndexingConfiguration",
    "code": "ResourceNotFoundException",
    "message": "Index AWS_Things does not exist. Please enable index by calling UpdateIndexingConfiguration",
    "time": "2021-07-31T19:45:12.415Z",
    "requestId": "4edf8af8-b11c-4b3e-8669-a2d3131214eb",
    "statusCode": 404,
    "retryable": false,
    "retryDelay": 84.26478306863561,
    "stack": [
        "ResourceNotFoundException: Index AWS_Things does not exist. Please enable index by calling UpdateIndexingConfiguration",
        "    at Object.extractError (/var/task/metrics.js:1:48361)",
        "    at constructor.extractError (/var/task/metrics.js:1:85054)",
        "    at constructor.callListeners (/var/task/metrics.js:1:105825)",
        "    at constructor.emit (/var/task/metrics.js:1:105535)",
        "    at constructor.emitEvent (/var/task/metrics.js:1:170852)",
        "    at constructor.e (/var/task/metrics.js:1:166359)",
        "    at r.runTo (/var/task/metrics.js:1:172694)",
        "    at /var/task/metrics.js:1:172900",
        "    at constructor.<anonymous> (/var/task/metrics.js:1:166629)",
        "    at constructor.<anonymous> (/var/task/metrics.js:1:170908)"
    ]
}

I was able to get rid of the above error by running the following from the AWS CLI

aws iot update-indexing-configuration --thing-indexing-configuration "thingIndexingMode=REGISTRY_AND_SHADOW, thingConnectivityIndexingMode=STATUS"

No idea if it's ideal, but it got rid of the error. Though I don't think this has anything to do with the issue at hand. Just needed to be sure. May need to be added to the instructions.

I wasn't seeing anything useful in the logs but I turned the LOG_LEVEL up to debug and this is what I see in /aws/lambda/virtual-smart-home-dev-provision:

START RequestId: f56b1cbd-b620-4b34-940d-XXXXXX Version: $LATEST
2021-07-31T22:53:08.829Z    f56b1cbd-b620-4b34-940d-XXXXXX  ERROR   ● /check_version with query: 
{
    "nr_version": "2.0.3",
    "thingId": "vsht-0edb4d9f-17c4-446f-XXX-XXXXXXXX",
    "version": "2.12.0"
}

2021-07-31T22:53:08.886Z    f56b1cbd-b620-4b34-940d-XXXXXX  ERROR   ● RESPONSE: 
{
    "isAllowedVersion": true,
    "isLatestVersion": true,
    "updateHint": "",
    "allowedDeviceCount": 100
}

END RequestId: f56b1cbd-b620-4b34-XXX-XXXXXX

In the backchannel log:

START RequestId: 78b3947c-c66c-4a19-974f-1e7e1c989383 Version: $LATEST
2021-07-31T22:53:12.314Z    78b3947c-c66c-4a19-974f-1e7e1c989383    ERROR   ℹ REQUEST STUB: 
{
    "rule": "requestConfig",
    "thingId": "vsht-0edb4d9f-17c4-446f-XXX-XXXXXX",
    "vshVersion": "2.12.0"
}

2021-07-31T22:53:12.354Z    78b3947c-c66c-4a19-974f-1e7e1c989383    ERROR   ● REQ: 
{
    "rule": "requestConfig",
    "thingId": "vsht-0edb4d9f-17c4-446f-XXX-XXXXXX",
    "vshVersion": "2.12.0"
}

2021-07-31T22:53:12.900Z    78b3947c-c66c-4a19-974f-1e7e1c989383    ERROR   ● RESULT: true
END RequestId: 78b3947c-c66c-4a19-974f-1e7e1c989383

START RequestId: 5b719680-3bdd-4f9f-83da-899fbe85cee7 Version: $LATEST
2021-07-31T22:53:13.236Z    5b719680-3bdd-4f9f-83da-899fbe85cee7    ERROR   ℹ REQUEST STUB: 
{
    "rule": "bulkdiscover",
    "thingId": "vsht-0edb4d9f-17c4-446f-XXX-XXXXXXX",
    "vshVersion": "?.?.?"
}

2021-07-31T22:53:13.237Z    5b719680-3bdd-4f9f-83da-899fbe85cee7    ERROR   ● REQ: 
{
    "rule": "bulkdiscover",
    "thingId": "vsht-0edb4d9f-17c4-446f-XXX-XXXXXXX",
    "devices": [
        {
            "deviceId": "vshd-a47cb1XXXXXXX",
            "friendlyName": "bulby6",
            "template": "DIMMABLE_LIGHT_BULB"
        }
    ]
}

2021-07-31T22:53:13.348Z    5b719680-3bdd-4f9f-83da-899fbe85cee7    ERROR   ● ADD-OR-UPDATE-REPORT: {
  "event": {
    "header": {
      "namespace": "Alexa.Discovery",
      "name": "AddOrUpdateReport",
      "payloadVersion": "3",
      "messageId": "4bd882a9-a83b-4a62-90a9-e1bf94ffb9ea"
    },
    "payload": {
      "endpoints": [
        {
          "endpointId": "vshd-XXXXXXXXXXX",
          "manufacturerName": "virtual smart home",
          "description": "virtual dimmable light bulb: bulby6",
          "friendlyName": "bulby6",
          "cookie": [Object],
          "additionalAttributes": [Object],
          "displayCategories": [Array],
          "capabilities": [Array]
        }
      ],
      "scope": {
        "type": "BearerToken",
        "token": "[TOKEN DELETED]"
      }
    }
  }
}
2021-07-31T22:53:13.555Z    5b719680-3bdd-4f9f-83da-899fbe85cee7    ERROR   ● RESULT: true
END RequestId: 5b719680-3bdd-4f9f-83da-899fbe85cee7
csuermann commented 2 years ago

Hi @turbo2ltr,

This is helpful feedback! Thank you!

Regarding getEventGatewayUrl() I believe my code complies with the docs. It puzzles me that you could make it work by using the same URL for us-east-1 and us-west-2. It might have to do with not setting the FunctionArn correctly in step 23?! If you want your skill available globally (like I did), one certainly needs to provide respective FunctionArn for all Alexa regions.

Changing the main deployment region (to something other than eu-west-1) should only require updating the VSH_IOT_REGION environment variable in serverless.yml.. I will fix a few hardcoded region strings accordingly.

Thanks for reminding me of the missing IoT index configuration. It is not strictly needed but will certainly silence the errors in the metrics log group.

The devices are indeed now discovered in the app but immediately show unresponsive in the app. They don't get updated and trying to control them does not produce any messages in the node-red cosole. This will have to wait for another day.

This could stem from rate limiting. Since you're running your own backend, you could be more generous than me. I needed to introduce a way to throttle traffic, because some users really do crazy stuff with VSH, which produces millions of messages, which I end up paying the bill for.

turbo2ltr commented 2 years ago 
When you send a message to the Alexa event gateway, send it to the event endpoint 
that aligns with the geographic availability of your smart home skill.

North America: https://api.amazonalexa.com/v3/events
Europe: https://api.eu.amazonalexa.com/v3/events
Far East: https://api.fe.amazonalexa.com/v3/events

I read that as if you are using us-east-1 or us-west-2 (both in North America) it should go to https://api.amazonalexa.com/v3/events

I would deduce that the only time you'd use https://api.fe.amazonalexa.com/v3/events is if you had a lambda in one of the Asia Pacific regions(?).

Though I have to say that my interpretation is opposite if what this guy said worked. https://amazon.developer.forums.answerhub.com/questions/204230/correct-regional-endpoint-to-avoid-skill-disabled.html

I put it back so us-west-2 points to the FE endpoint and discovery stopped working. Can't argue with that.

I also found this which seems like it's saying to use a lambda in the us-west-2 region for the Far east. https://developer.amazon.com/en-US/docs/alexa/custom-skills/host-a-custom-skill-as-an-aws-lambda-function.html

Oddly, it also says you CAN use a lambda in us-west-1 which I read elsewhere you could not.

Anyway, am I correct in my thinking that if the discovery is working but the updating is not, that this is the issue? image

When I add or remove a new node and redeploy node-red, in the NodeRed console, I see messages being published to topics like $aws/things/vsht-0edb4d9f-17c4-446f-a526-XXXXXXXX/shadow/get/accepted

I see this activity in the Iot Thing Activity tab. I see the Shadow Document getting updated.

When I turn a device on or off, I see messages being published to (only) vsh/vsht-0edb4d9f-17c4-446f-a52 6-XXXXXXXXX/changeReport

I see them coming into Cloudwatch.

START RequestId: bada08ff-d9f2-4f98-9d30-e89e579edf15 Version: $LATEST
2021-08-01T16:38:46.045Z    bada08ff-d9f2-4f98-9d30-e89e579edf15    ERROR   ℹ REQUEST STUB: 
{
    "rule": "changeReport",
    "template": "DIMMABLE_LIGHT_BULB",
    "thingId": "vsht-0edb4d9f-17c4-446f-a526-XXXXXXXX",
    "endpointId": "vshd-5e034XXXXXXXX",
    "causeType": "PHYSICAL_INTERACTION",
    "vshVersion": "2.12.0"
}

2021-08-01T16:38:46.079Z    bada08ff-d9f2-4f98-9d30-e89e579edf15    ERROR   ● REQ: 
{
    "rule": "changeReport",
    "thingId": "vsht-0edb4d9f-17c4-446f-a526-XXXXXXXX",
    "template": "DIMMABLE_LIGHT_BULB",
    "endpointId": "vshd-5e0340XXXXXXXX",
    "properties": [
        {
            "namespace": "Alexa.BrightnessController",
            "name": "brightness",
            "value": 100,
            "changed": false
        },
        {
            "namespace": "Alexa.PowerController",
            "name": "powerState",
            "value": "OFF",
            "changed": true
        }
    ],
    "correlationToken": "",
    "causeType": "PHYSICAL_INTERACTION",
    "vshVersion": "2.12.0",
    "userIdToken": "XXXXXXXX"
}

2021-08-01T16:38:46.541Z    bada08ff-d9f2-4f98-9d30-e89e579edf15    ERROR   ● CHANGE REPORT: {
  "event": {
    "header": {
      "namespace": "Alexa",
      "name": "ChangeReport",
      "messageId": "1e9d99ea-3578-4ae5-8950-89b0c7d44348",
      "payloadVersion": "3"
    },
    "endpoint": {
      "scope": {
        "type": "BearerToken",
        "token": "XXXXXXXX"
      },
      "endpointId": "vshd-5e03XXXXXXXX"
    },
    "payload": { "change": { "cause": { "type": "PHYSICAL_INTERACTION" }, "properties": [ [Object] ] } }
  },
  "context": {
    "properties": [
      {
        "namespace": "Alexa.BrightnessController",
        "name": "brightness",
        "value": 100,
        "timeOfSample": "2021-08-01T16:38:46.540Z",
        "uncertaintyInMilliseconds": 500
      },
      {
        "namespace": "Alexa.EndpointHealth",
        "name": "connectivity",
        "value": { "value": "OK" },
        "timeOfSample": "2021-08-01T16:38:46.540Z",
        "uncertaintyInMilliseconds": 250
      }
    ]
  }
}
2021-08-01T16:38:46.692Z    bada08ff-d9f2-4f98-9d30-e89e579edf15    ERROR   ● RESULT: true
END RequestId: bada08ff-d9f2-4f98-9d30-e89e579edf15

So this is where my understanding gets fuzzy (well it's always been fuzzy). I don't know the details, but to me, everything above seems to looks good. I just don't know how the lambda tells alexa of the update, or how I test that.

Is there a test event I could send to the virtual-smart-home-dev-skill lambda function that would tell us from the "alexa entrypoint" to node-red is working? I suspect the authentication needs will make sending manual events rather difficult.

csuermann commented 2 years ago

Hi @turbo2ltr,

Anyway, am I correct in my thinking that if the discovery is working but the updating is not, that this is the issue?

I think so. On the other hand RESULT: true in the CHANGE REPORT logs indicate that the Event Gateway did accept the request.

Could you point out again what is currently still not working on your end?

turbo2ltr commented 2 years ago

Sorry, I'll try to be less verbose..

What doesn't work is the Alexa side.

So the Alexa interactions are either not getting to the skill or the skill is not sending them to node-red. I was trying to determine if there is some test event I could send from Lambda to verify the connection from the skill back to node-red.

What is working...starting from the node-red side,

csuermann commented 2 years ago

You could try the following:

  1. Restart Node-RED (to ensure you don't suffer from being rate-limited)
  2. Ask Alexa to discover new devices
  3. Check the ...-skill CloudWatch Log Group for the Discover directive from Alexa to arrive
  4. Ask Alexa to switch one of your virtual devices on
  5. Again check the ...-skill log. Make sure to check it in the right region.
turbo2ltr commented 2 years ago

In the Lambda function section for that function, the trigger is set for Alexa smart home and the ID listed under Configuration -> Triggers matches the SkillID listed for the skill in the developer console.

I confirmed the Function ARN listed on the the function page also matches the one set for the default endpoint in the skill. (I do not have any geographical endpoints specified as it will not take the same ARN for "North America" stating it must be us-east-1)

Also, trying to turn a device on or off also does not produce any messaging in Cloudwatch. The app shows "device is unresponsive" Screenshot_20210802-145407_1

turbo2ltr commented 2 years ago

Damn dude. got it working.

Bottom line. Don't deploy to us-west-2.

I left my us-west-2 deployment and created a us-east-1 deployment. Without changing anything in the skill i did a discovery request and guess who shows up on the east side uninvited? The alexa discovery messages that are supposed to be going to us-west-2!

So even though I had the default endpoint (and ONLY endpoint) set to us-west-2, amazon was sending the messages from my alexa account to us-east-1. Would be nice if Amazon would mention that in the docs. The page even says the geographic regions are optional.

You can use west or east in for the default skill endpoint. You CAN'T use west as the North American endpoint. or you get an error that says the ARN doesn't match a regex expression that has "us-east" forced into it.

String instance with value "arn:aws:lambda:us-west-1:XXXXXXXX:function:virtual-smart-home-dev-skill" at property path "$.manifest.apis.smartHome.regions.NA.endpoint.uri" does not match the regular expression: "arn:aws:lambda:us-east-1:[0-9]{12}:function:[a-zA-Z0-9-_]+(/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})?(:[a-zA-Z0-9-_]+)?".

So seems you have to use an east region for North america. I put the east ARN in the default as well as NA and it took it without issue. So I will shut down my west deployment but at this points I've pointed node-red to east so should all be good...

So everything is working. Thank you for your help in this journey.. I learned a lot of stuff I didn't know before. Gotta love cloud computing. Same great apps, only on someone else's computer that you have no control over! What could go wrong. lol.

csuermann commented 2 years ago

Congratulations! Great that you got it working in the end!

turbo2ltr commented 2 years ago

BTW, the paypalme link in node-red requires a paypal account. Do you have a regular 'donate' link for paypal so I can pay with a credit card without a paypal account?

csuermann commented 2 years ago

Hi @turbo2ltr,

I have set up a GitHub sponsorship page, which you could use without PayPal.

Thank you for your support!

csuermann commented 2 years ago

I forgot: There is also this alternative way to donate via PayPal

turbo2ltr commented 2 years ago

Just a note to those that may read this, INDEXING AND SEARCH does not appear to have any free tier so you will get charged a tiny amount.

csuermann commented 2 years ago

Good catch. It is also not really needed to run VSH. I added a note to the docs.