Create a canonical URL for the Civic Sandbox

[DingoEatingFuzz]

The Civic Sandbox should be seen as another piece of the Civic Platform. For this reason, despite the Sandbox API being hosted with AWS Lambda, it should have URLs that are familiar to our family of services.

• service.civicpdx.org/civic-sandbox: Home for the Sandbox API. Currently nothing, but it could be API docs in the future.
• service.civicpdx.org/civic-sandbox/api/: Reserved for an OpenAPI self-documenting API response
• service.civicpdx.org/civic-sandbox/api/packages: Redirects to the civicSandboxPackages Lambda function

I'm not sure what the best AWS service for creating this routing rule is, so this ticket is equal parts research and implementation.

cc @AraOshin

[iant01]

Seems a lot of lamda functions and aws step funtions are fronted by aws api gateway but would need to look into how that plays with the civicpdx.org domain and sub domains.

This is good that there is a lamda example in hacko now. This can be used in the future to look at a more cost effective implamentation than containers/ecs (at least for smaller less compute/memory items that can be split off into simpler/descrete functions. (Future cost/performace project after this seasons projects)

[MikeTheCanuck]

Notes from discussion with Ian:

• Requires us to setup API Gateway to point to the Lambda function
• then setup "custom path" in API Gateway
• "custom paths" require a TLS certificate

There may be a "direct-to-custom-path" approach that doesn't use API Gateway, but we're not sure that's viable here.

[DingoEatingFuzz]

API Gateway is already set up and pointing to the lambda function, so it sounds like the bulk of the work remaining is getting a tls cert.

There are two reasonable paths forward:

Option 1: Use AWS Certificate Manager

• Reading material: https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-custom-domains.html
• Pro: Probably the easiest path forward, since it's AWS to AWS
• Con: Certs cost money per month

Option 2: Use Let's Encrypt

• Reading material: https://medium.com/merapar/how-to-use-aws-api-gateway-with-lets-encrypt-ce60c5daed1a
• The blog post mentions that AWS Certificate Manager doesn't work with API Gateway, that' no longer true.
• Pro: Free
• Con: Probably more involved and will require some scripting

If I read the pricing for cert manager correctly, it will cost us $0.75 per month per cert. Which is to say it will cost us $0.75 per month.

Given that low cost, I'm not opposed to it, but I do think Let's Encrypt shares values with Hack Oregon.

I'll leave the call up to the devops team, but I would like to at least explore what it will take to also use tls for our frontends and our other APIs. There is no private data being transferred, but https is the new norm.

[MikeTheCanuck]

Thank you for the specifics Michael, that’ll make this a lot less of a jungle hunt.

I’m gonna vote in favour of “working first, good second”:

• let’s spend the buck and get the AWS cert up ASAP, so we’re in place for Demo Day no matter what
• then with whatever time we have to spare, we can swap in a Let’s Encrypt cert (I’ve been dying for a real reason to try them out myself)

If no one else is itching to take this on I’ll grab this on the weekend.

[MikeTheCanuck]

Checklist of possible things to do - updated to the new domain name:

• [ ] Route sandbox.civicpdx.org/civic-sandbox to (something - a React package running somewhere?)
• [ ] Route sandbox.civicpdx.org/civic-sandbox/api/ to (something - a 'swaggerified' Django app?)
• [ ] Configure DNS to map from sandbox.civicpdx.org to the API Gateway - "You must set up a DNS record to map the custom domain name to the regional domain name for API requests bound for the custom domain name to be routed to API Gateway through the mapped regional API endpoint"
• [ ] Route sandbox.civicpdx.org/civic-sandbox/api/packages to the civicSandboxPackages Lambda function
• [ ] Obtain TLS cert for sandbox.civicpdx.org
• [ ] Apply TLS cert to (ALB?)
• [ ] Reconfigure API Gateway to (use the cert)
• [ ] (anything else?)

[MikeTheCanuck]

Obtain Cert for service.civicpdx.org

• Followed the steps in the ACM console for Request a certificate
• requested cert for service.civicpdx.org
• attempted DNS validation
• created a new CNAME record in our service.civicpdx.org zone with the following attributes:
  • Name = _cb1858cb4845a5bd6299c366697a1ec0.service.civicpdx.org
  • Type = CNAME
  • Alias = No
  • TTL (Seconds) = 300
  • Value = _38ff555bcc34234c10565d6e24dff985.acm-validations.aws.
  • Routing Policy = Simple

It took 5-10 minutes to validate after that record was created, but finally the ACM status for that cert flipped to Issued

[MikeTheCanuck]

Configuring DNS

My read of the AWS Lambda docs (e.g. Set up Custom Domain for an API in API Gateway) so far indicates that when setting up a custom domain name, the entire domain name gets directed to the Lambda side of the house (API Gateway I believe).

Looking in our current Route 53 config for service.civicpdx.org it looks like that domain is already being redirected to our ECS cluster - this is the A record value: dualstack.hacko-integration-658279555.us-west-2.elb.amazonaws.com.

Conclusion

I believe that we'll have to setup a separate custom domain (e.g. lambda.civicpdx.org or functions.civicpdx.org) to be able to call the Sandbox' Lambda function with a URL better than the usual nightmare-long value issued by AWS.

[MikeTheCanuck]

Michael has recommended sandbox.civicpdx.org

[MikeTheCanuck]

Route 53 config for sandbox.civicpdx.org

We'll need:

• A record
• destination URL

And there's instructions for setting this up in the Custom Domain Name instructions:

• first request and validate the TLS cert - or else the Create Custom Domain Name request will fail with "Error: The provided certificate does not exist."
  • What made my head hurt was requesting the CNAME record in the sandbox.civicpdx.org namespace without having an A record for that namespace first. Turns out Route 53 doesn't do any more validation than necessary - it doesn't care if you create a CNAME for a namespace that doesn't have an A record. (DNS is not for the uninitiated 'round these parts.) And it further turns out that ACM really only needs that CNAME record to validate domain ownership - even though there's no associated A record.
• then we can follow the Set up a Regional Custom Domain Name Using the API Gateway Console instructions, whose successful result will respond with a Target Domain Name
  • in our case, it returned d-vih7gfr49i.execute-api.us-west-2.amazonaws.com
  • it also complained about the Base Path Mapping (I submitted "civic-sandbox/api/packages" and it responded with "Error: A base path may contain only letters, numbers and one of $-_.+!*'(),")
  • tried again with just Path = "civic-sandbox" (and Destination = "Sandbox v1", Stage = "production")
• then went over to Route 53 and tried to create an A record whose Name = "sandbox.civicpdx.org", Type = A, Alias = Yes, Alias Target = "d-vih7gfr49i.execute-api.us-west-2.amazonaws.com", and defaults for the other values
  • However, the Route 53 console doesn't yet recognize the "execute-api" namespace as a valid Alias target, so according to this article on Stackoverflow you have to make this update via the CLI
  • this is the specific command I ran:

    aws route53 change-resource-record-sets  --hosted-zone-id Z1PP2R4G54X1EA --change-batch file://path/to/your/setup-dns-record.json

  • with this setup-dns-record.json batch file:

    {
    "Changes": [
    {
    "Action": "CREATE",
    "ResourceRecordSet": {
    "Name": "sandbox.civicpdx.org",
    "Type": "A",
    "AliasTarget": {
      "DNSName": "d-vih7gfr49i.execute-api.us-west-2.amazonaws.com",
      "HostedZoneId": "Z2OJLYMUO9EFXC",
      "EvaluateTargetHealth": false
    }
    }
    }
    ]
    }

  • whose output when successful looks like:

    {
    "ChangeInfo": {
    "Status": "PENDING",
    "SubmittedAt": "2018-06-10T19:34:51.914Z",
    "Id": "/change/CR2C5UA9N7KQN"
    }
    }

  • NOTE: the --hosted-zone-id is the value for the HackO account (look at the Route 53 console > Hosted Zones > find the value under the Hosted Zone ID for the civicpdx.org zone), and the HostedZoneId in the JSON is the value from the Amazon API Gateway

End result

The following URI: https://sandbox.civicpdx.org/civic-sandbox

successfully responds with good JSON:

{
statusCode: 200,
body: {
packages: {
transportation: {
description: "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
foundations: [
"103",
"105",
"104"
],
default_foundation: "103",
slides: [
"102",
"106",
"001",
"002"
],
default_slide: "106"
},
affordable housing: {
description: "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
foundations: [
"105",
"104",
"006"
],
default_foundation: "105",
slides: [
"101",
"102",
"003",
"004"
],
default_slide: "101"
},
neighborhoods: {
description: "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
foundations: [
"104",
"006"
],
default_foundation: "006",
slides: [
"101",
"102",
"106",
"005"
],
default_slide: "102"
}
},
slides: {
101: {
name: "sweeps",
endpoint: "https://0uv7y2d29i.execute-api.us-east-2.amazonaws.com/mockslide101/",
description: "This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "ScreenGridMap"
},
102: {
name: "bus stops",
endpoint: "https://hsqxipv6u8.execute-api.us-east-2.amazonaws.com/mockslide102/",
description: "This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "ScatterPlotMap"
},
106: {
name: "bike lanes",
endpoint: "https://cekaghbj2k.execute-api.us-east-2.amazonaws.com/mockslide106",
description: "This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "PathMap"
},
001: {
name: "bike parking",
endpoint: "http://service.civicpdx.org/neighborhood-development/sandbox/slides/bikeparking/",
description: "Bike Parking: This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "ScatterPlotMap"
},
002: {
name: "bike lanes",
endpoint: "http://service.civicpdx.org/neighborhood-development/sandbox/slides/bikelanes/",
description: "Bike Lanes: This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "PathMap"
},
003: {
name: "parks",
endpoint: "http://service.civicpdx.org/neighborhood-development/sandbox/slides/parks/",
description: "Parks: This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "PolygonPlotMap"
},
004: {
name: "multi-use trails",
endpoint: "http://service.civicpdx.org/neighborhood-development/sandbox/slides/multiusetrails/",
description: "Multi-use Trails: This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "PathMap"
},
005: {
name: "community gardens",
endpoint: "http://service.civicpdx.org/neighborhood-development/sandbox/slides/communitygardens/",
description: "Community Gardens: This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "SmallPolygonMap"
}
},
foundations: {
103: {
name: "zip codes",
endpoint: "https://ctyhsin0r2.execute-api.us-east-2.amazonaws.com/mockfoundation103",
description: "This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "ChoroplethMap",
attributes: {
primary: {
field: "happy_index",
name: "average happy index rating"
},
secondary: {
field: "parks",
name: "number of parks"
}
}
},
104: {
name: "neighborhoods",
endpoint: "https://rnc45keyjk.execute-api.us-east-2.amazonaws.com/mockfoundation104",
description: "This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "ChoroplethMap",
attributes: {
primary: {
field: "name",
name: "neighborhood name"
},
secondary: {
field: null,
name: null
}
}
},
105: {
name: "voter precincts",
endpoint: "https://qfvf1wpc3l.execute-api.us-east-2.amazonaws.com/mockfoundation105",
description: "This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description. This is a fake description.",
visualization: "ChoroplethMap",
attributes: {
primary: {
field: "avg_voter_turnout_pct",
name: "average voter turnout"
},
secondary: {
field: "median_income",
name: "median income"
}
}
},
006: {
name: "Property Value",
endpoint: "http://service.civicpdx.org/neighborhood-development/sandbox/foundations/taxlotblockgroups/",
description: "Property Value: This is a fake description. This is a fake description. This is a fake description. This is a fake description. ",
visualization: "ChoroplethMap"
}
}
}
}

[MikeTheCanuck]

Reconfiguring API Gateway for intended routing

Goal: setup the requested /civic-sandbox/api/packages route to answer with the Lambda function, rather than answering from /civic-sandbox as is does right now.

(naive) Steps taken

• select the API defined as "Sandbox-v1"
• on the Actions button, select "Create Resource"
• give it a Resource Path of "api", and enable "Enable API Gateway CORS"
• on the Actions button, select "Create Method"
• choose a "GET" method, and in the Setup panel, choose an Integration type of "Lamdba function", leave Lambda region as "us-west-2", and set Lambda function to "arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk" (which is the ARN of "Sandbox-v1")
• Note: the API Gateway console throws this warning: "You are about to give API Gateway permission to invoke your Lambda function: arn:aws:lambda:us-west-2:845828040396:function:arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk"
• Note: this attempt received the error "1 validation error detected: Value 'arn:aws:lambda:us-west-2:845828040396:function:arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk' at 'functionName' failed to satisfy constraint: Member must satisfy regular expression pattern: (arn:(aws[a-zA-Z-]*)?:lambda:)?([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1}:)?(\d{12}:)?(function:)?([a-zA-Z0-9-]+)(:(\$LATEST|[a-zA-Z0-9-]+))?"

Comparing test results between configured GET mappings

• When clicking the Test button in the "/ - GET - Method Execution" panel, it returns the following

  Execution log for request 87463648-6ceb-11e8-ba1a-35a46d27507c
  Sun Jun 10 20:19:07 UTC 2018 : Starting execution for request: 87463648-6ceb-11e8-ba1a-35a46d27507c
  Sun Jun 10 20:19:07 UTC 2018 : HTTP Method: GET, Resource Path: /
  Sun Jun 10 20:19:07 UTC 2018 : Method request path: {}
  Sun Jun 10 20:19:07 UTC 2018 : Method request query string: {}
  Sun Jun 10 20:19:07 UTC 2018 : Method request headers: {}
  Sun Jun 10 20:19:07 UTC 2018 : Method request body before transformations: 
  Sun Jun 10 20:19:07 UTC 2018 : Endpoint request URI: https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:civicSandboxPackages/invocations
  Sun Jun 10 20:19:07 UTC 2018 : Endpoint request headers: {x-amzn-lambda-integration-tag=87463648-6ceb-11e8-ba1a-35a46d27507c, Authorization=**************************************************************************************************************************************************************************************************************************************************************************************05224d, X-Amz-Date=20180610T201907Z, x-amzn-apigateway-api-id=43iqjy2gjk, X-Amz-Source-Arn=arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk/test-invoke-stage/GET/, Accept=application/json, User-Agent=AmazonAPIGateway_43iqjy2gjk, X-Amz-Security-Token=FQoDYXdzEB0aDGRYk21vDhnXiEnqyCK3AwcjWELWm+IuXhcsdFO6+KLwLLuK7200+sdM3tFc8H591odBeQ3SniDF6zl/pZw3EKuoRSznXUYs15I8PETyZox7kegk2J6WGiqpAoEQwvgds/C3080OOlebwAST8O3FRIw6Y+OGTO7g505lKM27vLVmRH7JWr1mdrbjOPaVGW5uqCAY5hSu/CHr+vuzsVnGpblOxj6CCFZ9dByZeiLRE8e+7EUCKMUIh70qn3ORJ5IyuLsCuxA5YJGu+Iv6qvHWHw/LveT4EMt6XWIpML0zlMK4MvpOxnwCdc+hyQfEI8S09gVtQgWTk5mRR4+DqJuXS5XTP7yGLf34Jnm+QCYsr/R9WVW0OS [TRUNCATED]
  Sun Jun 10 20:19:07 UTC 2018 : Endpoint request body after transformations: 
  Sun Jun 10 20:19:07 UTC 2018 : Sending request to https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:civicSandboxPackages/invocations
  Sun Jun 10 20:19:07 UTC 2018 : Received response. Integration latency: 245 ms
  Sun Jun 10 20:19:07 UTC 2018 : Endpoint response body before transformations: {"statusCode": 200, "body": {"packages": {"transportation": {"description": "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.", "foundations": ["103", "105", "104"], "default_foundation": "103", "slides": ["102", "106", "001", "002"], "default_slide": "106"}, "affordable housing": {"description": "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.", "foundations": ["105", "104", "006"], "default_foundation": "105", "slides": ["101", "102", "003", "004"], "default_slide": "101"}, "neighborhoods": {"description": "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.", "foundations": ["104", "006"], "default_foundation": "006", "slides": ["101", "102", "106", "005"], "default_slide": "102"}}, "slides": {"001": {"name": "bike parking", "end [TRUNCATED]
  Sun Jun 10 20:19:07 UTC 2018 : Endpoint response headers: {X-Amz-Executed-Version=$LATEST, x-amzn-Remapped-Content-Length=0, Connection=keep-alive, x-amzn-RequestId=87479629-6ceb-11e8-a048-79fdeab69ef2, Content-Length=5847, Date=Sun, 10 Jun 2018 20:19:07 GMT, X-Amzn-Trace-Id=root=1-5b1d87bb-77e4919a70765cb35d547f84;sampled=0, Content-Type=application/json}
  Sun Jun 10 20:19:07 UTC 2018 : Method response body after transformations: {"statusCode": 200, "body": {"packages": {"transportation": {"description": "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.", "foundations": ["103", "105", "104"], "default_foundation": "103", "slides": ["102", "106", "001", "002"], "default_slide": "106"}, "affordable housing": {"description": "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.", "foundations": ["105", "104", "006"], "default_foundation": "105", "slides": ["101", "102", "003", "004"], "default_slide": "101"}, "neighborhoods": {"description": "Transportation: This is a fake description. This is a fake description. This is a fake description. This is a fake description.", "foundations": ["104", "006"], "default_foundation": "006", "slides": ["101", "102", "106", "005"], "default_slide": "102"}}, "slides": {"001": {"name": "bike parking", "endpoi [TRUNCATED]
  Sun Jun 10 20:19:07 UTC 2018 : Method response headers: {Access-Control-Allow-Origin=*, X-Amzn-Trace-Id=Root=1-5b1d87bb-77e4919a70765cb35d547f84, Content-Type=application/json}
  Sun Jun 10 20:19:07 UTC 2018 : Successfully completed execution
  Sun Jun 10 20:19:07 UTC 2018 : Method completed with status: 200

• When clicking the Test button in the "/api - GET - Method Execution" panel, it returns the following

  
  Execution log for request e6a41f57-6ceb-11e8-ba66-c7fdc9f70b8c
  Sun Jun 10 20:21:47 UTC 2018 : Starting execution for request: e6a41f57-6ceb-11e8-ba66-c7fdc9f70b8c
  Sun Jun 10 20:21:47 UTC 2018 : HTTP Method: GET, Resource Path: /api
  Sun Jun 10 20:21:47 UTC 2018 : Method request path: {}
  Sun Jun 10 20:21:47 UTC 2018 : Method request query string: {}
  Sun Jun 10 20:21:47 UTC 2018 : Method request headers: {}
  Sun Jun 10 20:21:47 UTC 2018 : Method request body before transformations: 
  Sun Jun 10 20:21:47 UTC 2018 : Endpoint request URI: https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk/invocations
  Sun Jun 10 20:21:47 UTC 2018 : Endpoint request headers: {x-amzn-lambda-integration-tag=e6a41f57-6ceb-11e8-ba66-c7fdc9f70b8c, Authorization=**************************************************************************************************************************************************************************************************************************************************************************************e79cfc, X-Amz-Date=20180610T202147Z, x-amzn-apigateway-api-id=43iqjy2gjk, X-Amz-Source-Arn=arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk/test-invoke-stage/GET/api, Accept=application/json, User-Agent=AmazonAPIGateway_43iqjy2gjk, X-Amz-Security-Token=FQoDYXdzEBwaDLuw3V9bOsZ2EzbA1yK3A4cP1ReSYk5jOO1nM1YoyM8ArF9pFfxnYYWvxYRsaZDi66TgsCbKx8BIdXpiV8NGwnxtYvy0dVlwoGRiBeB8lNJWapn0MhnPeDUkf0Tsq0pZ/XTf5LFxU2cJ1VTFyfKyfaOhJypW8ro08E2EJAVovRpeOIR2i+w0/8YpV51qPdenyJYfVgSEeEjwchOL/PY0yjq+8ezzWX3melKvL29O5hgMTPDkB6HzSXR0AqlYdv6x5SHChqn+AIE3Xmuq5mQzrFp5aI5zjjWpdPFhomgWUhrNfiFQ9B/o2dvDmjuG6iG4giB+/Zxb3I2zXZWq1KBXLu4by1XdTE7921zN5oE4AdHajaS [TRUNCATED]
  Sun Jun 10 20:21:47 UTC 2018 : Endpoint request body after transformations: 
  Sun Jun 10 20:21:47 UTC 2018 : Sending request to https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk/invocations
  Sun Jun 10 20:21:47 UTC 2018 : Received response. Integration latency: 5 ms
  Sun Jun 10 20:21:47 UTC 2018 : Endpoint response body before transformations: {"message":"1 validation error detected: Value 'arn:aws:lambda:us-west-2:845828040396:function:arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk' at 'functionName' failed to satisfy constraint: Member must satisfy regular expression pattern: (arn:(aws[a-zA-Z-]*)?:lambda:)?([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_\\.]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?"}

Sun Jun 10 20:21:47 UTC 2018 : Endpoint response headers: {Connection=keep-alive, x-amzn-RequestId=e6a49424-6ceb-11e8-b236-0ddc2d935ba0, x-amzn-ErrorType=ValidationException, Content-Length=398, Date=Sun, 10 Jun 2018 20:21:47 GMT, Content-Type=application/json} Sun Jun 10 20:21:47 UTC 2018 : Lambda invocation failed with status: 400 Sun Jun 10 20:21:47 UTC 2018 : Execution failed: 1 validation error detected: Value 'arn:aws:lambda:us-west-2:845828040396:function:arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk' at 'functionName' failed to satisfy constraint: Member must satisfy regular expression pattern: (arn:(aws[a-zA-Z-]*)?:lambda:)?([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1}:)?(\d{12}:)?(function:)?([a-zA-Z0-9-.]+)(:(\$LATEST|[a-zA-Z0-9-]+))? Sun Jun 10 20:21:47 UTC 2018 : Method completed with status: 400

- THE DIFFERENCE: 
  - Good URI: https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:civicSandboxPackages/invocations
  - Bad URI:    https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk/invocations
- Thus, I deleted the `/api` GET entry and created it with Lambda function = "arn:aws:lambda:us-west-2:845828040396:function:civicSandboxPackages/invocations"
- which throws a better-looking warning "You are about to give API Gateway permission to invoke your Lambda function: arn:aws:lambda:us-west-2:845828040396:function:civicSandboxPackages/invocations"
- but which then throws an error "An unknown error occurred"
- and when I test that with the "/api GET" Test function, returns an execution log of:

Execution log for request 0443eef3-6ced-11e8-9e05-0beac8d1f5d2 Sun Jun 10 20:29:46 UTC 2018 : Starting execution for request: 0443eef3-6ced-11e8-9e05-0beac8d1f5d2 Sun Jun 10 20:29:46 UTC 2018 : HTTP Method: GET, Resource Path: /api Sun Jun 10 20:29:46 UTC 2018 : Method request path: {} Sun Jun 10 20:29:46 UTC 2018 : Method request query string: {} Sun Jun 10 20:29:46 UTC 2018 : Method request headers: {} Sun Jun 10 20:29:46 UTC 2018 : Method request body before transformations: Sun Jun 10 20:29:46 UTC 2018 : Endpoint request URI: https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:civicSandboxPackages/invocations/invocations Sun Jun 10 20:29:46 UTC 2018 : Endpoint request headers: {x-amzn-lambda-integration-tag=0443eef3-6ced-11e8-9e05-0beac8d1f5d2, Authorization=**7269b0, X-Amz-Date=20180610T202946Z, x-amzn-apigateway-api-id=43iqjy2gjk, X-Amz-Source-Arn=arn:aws:execute-api:us-west-2:845828040396:43iqjy2gjk/test-invoke-stage/GET/api, Accept=application/json, User-Agent=AmazonAPIGateway_43iqjy2gjk, X-Amz-Security-Token=FQoDYXdzEB0aDNhCHBNH6tWnU+arPiK3A46CAVIAG58uLAkZ8+/WVi9FPnTAk5EXr9/YOPhu1L/eR0YsGQHoPXwzglRsKrWac4WPwvucMlyxsQbTkfAwkwQGa2OH/VCrbWbqVvjKdsC2qD95gWaYj73S1BTHUdWN+vJELTef1xOeCW1CACcMdQ1C753up66uXZBISg0zKVBmEU/tzLD/XURInCiR3JJ/J0RGJVM1nZdSYYlbk1N9YJf2aI4EF2QK/yzY0/hFq2k4Wii06oyQwp8xW2JnWIoomKbVS62aq/bMJhrylVR4HywTLZ54nAX3dj0DO4/REyAg6r8DT0Fbg8pKhQTg5p7MX4wmlCRrzbW19v+gVTIX/1bA1jd [TRUNCATED] Sun Jun 10 20:29:46 UTC 2018 : Endpoint request body after transformations: Sun Jun 10 20:29:46 UTC 2018 : Sending request to https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:845828040396:function:civicSandboxPackages/invocations/invocations Sun Jun 10 20:29:46 UTC 2018 : Received response. Integration latency: 2 ms Sun Jun 10 20:29:46 UTC 2018 : Endpoint response body before transformations:

Unable to determine service/operation name to be authorized

Sun Jun 10 20:29:46 UTC 2018 : Endpoint response headers: {Connection=keep-alive, x-amzn-RequestId=0444b245-6ced-11e8-845a-47dfea39f2c3, Content-Length=130, Date=Sun, 10 Jun 2018 20:29:46 GMT} Sun Jun 10 20:29:46 UTC 2018 : Lambda invocation failed with status: 403 Sun Jun 10 20:29:46 UTC 2018 : Execution failed due to configuration error: Sun Jun 10 20:29:46 UTC 2018 : Method completed with status: 500

### WORSE: 
This has somehow made the URI https://sandbox.civicpdx.org/civic-sandbox return a Forbidden message, so I've deleted the entire `/api` Resource from the AWS Gateway - but that still hasn't fixed the 403 Forbidden error we're now getting from that request (even though the internal Test - from the "/ GET" method in the API Gateway console - is receiving a 200 response).