Figure out how to deploy Datasette to AWS Lambda using function URLs and Mangum

[simonw]

Initial research and proof of concept for the datasette-publish-lambda plugin.

[simonw]

YouTube video from three months ago about deploying fast api on lambda https://youtu.be/RGIM4JfsSk0

Wow that video genuinely has everything I need to know - including how to make the zip file and how to deploy (via the console) and how to add a function URL

https://github.com/pixegami/fastapi-tutorial

from mangum import Mangum

app = FastAPI()
handler = Mangum(app)

pip install -t lib -r requirements.txt

(cd lib; zip ../lambda_function.zip -r .)

zip lambda_function.zip -u main.py
zip lambda_function.zip -u books.json

Looks like you need cloudfront if you want a custom domain or subdomain though: https://medium.com/@walid.karray/configuring-a-custom-domain-for-aws-lambda-function-url-with-serverless-framework-c0d78abdc253

(I bet cloudflare is easier)

Limit is still 50 MB (zipped, for direct upload) 250 MB (unzipped)

I want "datasette publish lambda"

I'm really tempted to have the datasette-publish-lambda GitHub repo build a release which is a zip file that has everything it needs - then the plugin itself downloads (and caches) the latest releases zip file, adds the SQLite DB and metadata and maybe some templates and plugins and static files to that zip file, adds any --install plugins to it and then ships the result

But the first release can build the zip file from scratch every time. That's probably simpler!

So the plugin mainly creates a working zip file for you and deploys it.

It could even have a --test option which creates the zip file, then unzips it into a temp virtual environment to test it before deploying it

Trickiest code here is the code that deploys the function. I'm VERY tempted to build a separate asgi-lambda-publish python package here that this depends on, because I'm so angry about how hard this is

Start with a TIL though.

[simonw]

Could I start by doing a full lambda deployment entirely using the AWS CLI tool?

[simonw]

https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-awscli.html is VERY useful.

aws iam create-role \
  --role-name lambda-ex \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"}
    ]}'

aws iam attach-role-policy \
  --role-name lambda-ex \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Then index.js:

exports.handler = async function(event, context) {
  console.log("ENVIRONMENT VARIABLES\n" + JSON.stringify(process.env, null, 2))
  console.log("EVENT\n" + JSON.stringify(event, null, 2))
  return context.logStreamName
}

Create function.zip:

zip function.zip index.js

Next step needs account ID:

export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text) 
echo "hello:${AWS_ACCOUNT_ID}:there"

Outputs:

hello:462092780466:there

To create function:

aws lambda create-function \
  --function-name my-test-lambda-nodejs-function \
  --zip-file fileb://function.zip \
  --handler index.handler \
  --runtime nodejs16.x \
  --role "arn:aws:iam::${AWS_ACCOUNT_ID}:role/lambda-ex"

Then invoke the function like this:

aws lambda invoke --function-name my-test-lambda-nodejs-function out --log-type Tail

[simonw]

https://docs.aws.amazon.com/lambda/latest/dg/urls-configuration.html#create-url-cli shows how to add a function URL afterwards.

[simonw]

Tried out those steps, got to:

/tmp % aws lambda create-function \
  --function-name my-test-lambda-nodejs-function \
  --zip-file fileb://function.zip \
  --handler index.handler \
  --runtime nodejs16.x \
  --role "arn:aws:iam::${AWS_ACCOUNT_ID}:role/lambda-ex"
{
    "FunctionName": "my-test-lambda-nodejs-function",
    "FunctionArn": "arn:aws:lambda:us-east-1:462092780466:function:my-test-lambda-nodejs-function",
    "Runtime": "nodejs16.x",
    "Role": "arn:aws:iam::462092780466:role/lambda-ex",
    "Handler": "index.handler",
    "CodeSize": 320,
    "Description": "",
    "Timeout": 3,
    "MemorySize": 128,
    "LastModified": "2022-09-18T17:44:07.957+0000",
    "CodeSha256": "swC+abUCZgQL7frXislbhKvIYkeClJjEGa9r7WwFTCg=",
    "Version": "$LATEST",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "62a67b98-7992-4fc9-97b5-653371bbaf28",
    "State": "Pending",
    "StateReason": "The function is being created.",
    "StateReasonCode": "Creating",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ]
}

This didn't work though:

/tmp % aws lambda invoke --function-name my-test-lambda-nodejs-function out --log-type Tail
{
    "StatusCode": 200,
    "FunctionError": "Unhandled",
    "LogResult": "MjAyMi0wOS0xOFQxNzo0NDoyNS45MzhaCXVuZGVmaW5lZAlFUlJPUglVbmNhdWdodCBFeGNlcHRpb24gCXsiZXJyb3JUeXBlIjoiUnVudGltZS5Vc2VyQ29kZVN5bnRheEVycm9yIiwiZXJyb3JNZXNzYWdlIjoiU3ludGF4RXJyb3I6IEludmFsaWQgb3IgdW5leHBlY3RlZCB0b2tlbiIsInN0YWNrIjpbIlJ1bnRpbWUuVXNlckNvZGVTeW50YXhFcnJvcjogU3ludGF4RXJyb3I6IEludmFsaWQgb3IgdW5leHBlY3RlZCB0b2tlbiIsIiAgICBhdCBfbG9hZFVzZXJBcHAgKGZpbGU6Ly8vdmFyL3J1bnRpbWUvaW5kZXgubWpzOjk0ODoxNykiLCIgICAgYXQgYXN5bmMgT2JqZWN0LlVzZXJGdW5jdGlvbi5qcy5tb2R1bGUuZXhwb3J0cy5sb2FkIChmaWxlOi8vL3Zhci9ydW50aW1lL2luZGV4Lm1qczo5NzY6MjEpIiwiICAgIGF0IGFzeW5jIHN0YXJ0IChmaWxlOi8vL3Zhci9ydW50aW1lL2luZGV4Lm1qczoxMTM3OjIzKSIsIiAgICBhdCBhc3luYyBmaWxlOi8vL3Zhci9ydW50aW1lL2luZGV4Lm1qczoxMTQzOjEiXX0KU1RBUlQgUmVxdWVzdElkOiBhMjBjZDhlOS1lYmFiLTRmNmUtOTMxOS1iZmY4NmNjZDExZmIgVmVyc2lvbjogJExBVEVTVAoyMDIyLTA5LTE4VDE3OjQ0OjI3LjI3OVoJdW5kZWZpbmVkCUVSUk9SCVVuY2F1Z2h0IEV4Y2VwdGlvbiAJeyJlcnJvclR5cGUiOiJSdW50aW1lLlVzZXJDb2RlU3ludGF4RXJyb3IiLCJlcnJvck1lc3NhZ2UiOiJTeW50YXhFcnJvcjogSW52YWxpZCBvciB1bmV4cGVjdGVkIHRva2VuIiwic3RhY2siOlsiUnVudGltZS5Vc2VyQ29kZVN5bnRheEVycm9yOiBTeW50YXhFcnJvcjogSW52YWxpZCBvciB1bmV4cGVjdGVkIHRva2VuIiwiICAgIGF0IF9sb2FkVXNlckFwcCAoZmlsZTovLy92YXIvcnVudGltZS9pbmRleC5tanM6OTQ4OjE3KSIsIiAgICBhdCBhc3luYyBPYmplY3QuVXNlckZ1bmN0aW9uLmpzLm1vZHVsZS5leHBvcnRzLmxvYWQgKGZpbGU6Ly8vdmFyL3J1bnRpbWUvaW5kZXgubWpzOjk3NjoyMSkiLCIgICAgYXQgYXN5bmMgc3RhcnQgKGZpbGU6Ly8vdmFyL3J1bnRpbWUvaW5kZXgubWpzOjExMzc6MjMpIiwiICAgIGF0IGFzeW5jIGZpbGU6Ly8vdmFyL3J1bnRpbWUvaW5kZXgubWpzOjExNDM6MSJdfQpFTkQgUmVxdWVzdElkOiBhMjBjZDhlOS1lYmFiLTRmNmUtOTMxOS1iZmY4NmNjZDExZmIKUkVQT1JUIFJlcXVlc3RJZDogYTIwY2Q4ZTktZWJhYi00ZjZlLTkzMTktYmZmODZjY2QxMWZiCUR1cmF0aW9uOiAxNTIzLjA2IG1zCUJpbGxlZCBEdXJhdGlvbjogMTUyNCBtcwlNZW1vcnkgU2l6ZTogMTI4IE1CCU1heCBNZW1vcnkgVXNlZDogMTIgTUIJClVua25vd24gYXBwbGljYXRpb24gZXJyb3Igb2NjdXJyZWQKUnVudGltZS5Vc2VyQ29kZVN5bnRheEVycm9yCg==",
    "ExecutedVersion": "$LATEST"
}

Ran that through base64 --decode like this:

/tmp % cat out.json | jq .LogResult -r | base64 --decode
Unknown application error occurred
Runtime.UserCodeSyntaxError
2022-09-18T17:44:54.675Z    undefined   ERROR   Uncaught Exception  {"errorType":"Runtime.UserCodeSyntaxError","errorMessage":"SyntaxError: Invalid or unexpected token","stack":["Runtime.UserCodeSyntaxError: SyntaxError: Invalid or unexpected token","    at _loadUserApp (file:///var/runtime/index.mjs:948:17)","    at async Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:976:21)","    at async start (file:///var/runtime/index.mjs:1137:23)","    at async file:///var/runtime/index.mjs:1143:1"]}
START RequestId: f71d6749-43af-4916-abe8-649249be31dd Version: $LATEST
2022-09-18T17:45:07.299Z    undefined   ERROR   Uncaught Exception  {"errorType":"Runtime.UserCodeSyntaxError","errorMessage":"SyntaxError: Invalid or unexpected token","stack":["Runtime.UserCodeSyntaxError: SyntaxError: Invalid or unexpected token","    at _loadUserApp (file:///var/runtime/index.mjs:948:17)","    at async Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:976:21)","    at async start (file:///var/runtime/index.mjs:1137:23)","    at async file:///var/runtime/index.mjs:1143:1"]}
END RequestId: f71d6749-43af-4916-abe8-649249be31dd
REPORT RequestId: f71d6749-43af-4916-abe8-649249be31dd  Duration: 1486.95 ms    Billed Duration: 1487 ms    Memory Size: 128 MB Max Memory Used: 12 MB  
Unknown application error occurred
Runtime.UserCodeSyntaxError

[simonw]

That's because I broke my index.js file:

/tmp % cat index.js 
exports.handler = async function(event, context) {
  console.log("ENVIRONMENT VARIABLES
" + JSON.stringify(process.env, null, 2))
  console.log("EVENT
" + JSON.stringify(event, null, 2))
  return context.logStreamName
}

Trying again with a fixed one.

[simonw]

I recreated the functions.zip file - I think this will update the function:

aws lambda update-function-code \
  --function-name my-test-lambda-nodejs-function \
  --zip-file fileb://function.zip

[simonw]

That seems to work!

aws lambda invoke --function-name my-test-lambda-nodejs-function out --log-type Tail

{
    "StatusCode": 200,
    "LogResult": "U1RBUlQgUmVxdWVzdElkOiBmYmMxMGJmMC0yZWZkLTRkYTctOTA0Yy1hNGE4MTAyYzA1MzMgVmVyc2lvbjogJExBVEVTVAoyMDIyLTA5LTE4VDE3OjQ4OjQ2LjA5N1oJZmJjMTBiZjAtMmVmZC00ZGE3LTkwNGMtYTRhODEwMmMwNTMzCUlORk8JRU5WSVJPTk1FTlQgVkFSSUFCTEVTCnsKICAiQVdTX0xBTUJEQV9GVU5DVElPTl9WRVJTSU9OIjogIiRMQVRFU1QiLAogICJBV1NfU0VTU0lPTl9UT0tFTiI6ICJJUW9KYjNKcFoybHVYMlZqRUlyLy8vLy8vLy8vL3dFYUNYVnpMV1ZoYzNRdE1TSkhNRVVDSUNkQmV5OTFYcnNvTjdWNmtoRkgwTkpsVXhuVy9pN0N2cDlGNitCS0NEUjlBaUVBc1k3VUMzZnF1ZXpwam93QmswRE5TTUlEY2pTeXY3Y1RUMkJYVFhCbjZqZ3FrUU1JTXhBREdndzBOakl3T1RJM09EQTBOallpREFzYXFlcE41ZG5Rb0N5aklTcnVBbEVtQVJIRTVzVkpKWTdkcGNmL2VhcXd3TGZDMUc4cmJ2OHdkY250dS9ZUVVnWjAreGtLbGZRSG9wU1JaV3NhQ3hyaFBqRitCVDRvUGg5Y0xYU2R2ZFhUS2VVOGM1c0k2NmNWVUtzT3drS2NXK2poNmp6THpvQXpJdkFyYlB4b3duL3BmSDBBQ2dxNGFsU0RVQWdyVWJiV3dSNk1Vb1FsOUhMUmd2VnFEWjlSbnR6OXpOcHcwU0pnS0lSWTRYZloxeFNQTGlnVGRmblRLcTVja1RBaFFDWEgrbFN4WW12aWlsRGJNUXdlYzJSczYvdzNJZ0I5MkFJcDJYdkVNL00wRnZuK01QbjRCMTQ5Q2piaFRCTmVGdlFlWkY3Wmd4b1Q1eWFveFRLOGFBL0YyTG9mQkt4RzF0akxyeDc5OXFwQ0JWZ2d4bkFPcVErT1dJTUFkVjZjYTFkNUFtUm5ybHdISUMzV3FkL1FwTGNBTTdTL3pvSU41Rk5VQm1WOXlMcG5FR2pVVVJLYU1Fa05xZld1MFVvZVZoaHVPY1hwdVZSR01zZENTMXR1amc2QmJYcXJWS0U5K05uZHd3UVVRa0h3R0tKcGlOa2VXWWIyYjJENlp4WmZ5VGpKMHRqc21jZGZmcTlNL05DRlJqRDZzNTJaQmpxZEFYR1VVTnBabWFQSWMvNk4yZ21BNWZlTHpyZml2UEhaVEM0K3VqMDI2VFNLRUJsa2RPTTk3N2RPbzBMTGJPL3grdWUzRXZOWTZVa1N1ek9kQktRNWVuNTJwVXBQUmd6RTVvWUI1WFlHMVJZc0xaVkx6di9lNzNmaHROemtvVFpkdGdiK2FSWUdXbzczZHoxUkM3NnJ1NTlvdlAwSE9YNkU1bXRtTm1lWnhRU1BvRDVQamRxUzhYQWQ5K1RwMDBBRGdncjV3M1M5MkFWVFFqL2RncHc9IiwKICAiQVdTX0xBTUJEQV9MT0dfR1JPVVBfTkFNRSI6ICIvYXdzL2xhbWJkYS9teS10ZXN0LWxhbWJkYS1ub2RlanMtZnVuY3Rpb24iLAogICJMRF9MSUJSQVJZX1BBVEgiOiAiL3Zhci9sYW5nL2xpYjovbGliNjQ6L3Vzci9saWI2NDovdmFyL3J1bnRpbWU6L3Zhci9ydW50aW1lL2xpYjovdmFyL3Rhc2s6L3Zhci90YXNrL2xpYjovb3B0L2xpYiIsCiAgIkxBTUJEQV9UQVNLX1JPT1QiOiAiL3Zhci90YXNrIiwKICAiQVdTX0xBTUJEQV9SVU5USU1FX0FQSSI6ICIxMjcuMC4wLjE6OTAwMSIsCiAgIkFXU19MQU1CREFfTE9HX1NUUkVBTV9OQU1FIjogIjIwMjIvMDkvMTgvWyRMQVRFU1RdNzNkZWI3NGRkOGI1NDczNzhkNzQxOGQwMjVkZGE2NzAiLAogICJBV1NfRVhFQ1VUSU9OX0VOViI6ICJBV1NfTGFtYmRhX25vZGVqczE2LngiLAogICJBV1NfTEFNQkRBX0ZVTkNUSU9OX05BTUUiOiAibXktdGVzdC1sYW1iZGEtbm9kZWpzLWZ1bmN0aW9uIiwKICAiQVdTX1hSQVlfREFFTU9OX0FERFJFU1MiOiAiMTY5LjI1NC43OS4xMjk6MjAwMCIsCiAgIlBBVEgiOiAiL3Zhci9sYW5nL2JpbjovdXNyL2xvY2FsL2JpbjovdXNyL2Jpbi86L2Jpbjovb3B0L2JpbiIsCiAgIkFXU19ERUZBVUxUX1JFR0lPTiI6ICJ1cy1lYXN0LTEiLAogICJQV0QiOiAiL3Zhci90YXNrIiwKICAiQVdTX1NFQ1JFVF9BQ0NFU1NfS0VZIjogIlBXOXRrQWlibFNjQXJpUG14aHRNVWlrWVFvUmtLRDdGdmtLSG5pdGoiLAogICJMQU1CREFfUlVOVElNRV9ESVIiOiAiL3Zhci9ydW50aW1lIiwKICAiTEFORyI6ICJlbl9VUy5VVEYtOCIsCiAgIkFXU19MQU1CREFfSU5JVElBTElaQVRJT05fVFlQRSI6ICJvbi1kZW1hbmQiLAogICJOT0RFX1BBVEgiOiAiL29wdC9ub2RlanMvbm9kZTE2L25vZGVfbW9kdWxlczovb3B0L25vZGVqcy9ub2RlX21vZHVsZXM6L3Zhci9ydW50aW1lL25vZGVfbW9kdWxlczovdmFyL3J1bnRpbWU6L3Zhci90YXNrIiwKICAiVFoiOiAiOlVUQyIsCiAgIkFXU19SRUdJT04iOiAidXMtZWFzdC0xIiwKICAiQVdTX0FDQ0VTU19LRVlfSUQiOiAiQVNJQVdYRlhBSU9aTVlUVDZBTUciLAogICJTSExWTCI6ICIwIiwKICAiX0FXU19YUkFZX0RBRU1PTl9BRERSRVNTIjogIjE2OS4yNTQuNzkuMTI5IiwKICAiX0FXU19YUkFZX0RBRU1PTl9QT1JUIjogIjIwMDAiLAogICJBV1NfWFJBWV9DT05URVhUX01JU1NJTkciOiAiTE9HX0VSUk9SIiwKICAiX0hBTkRMRVIiOiAiaW5kZXguaGFuZGxlciIsCiAgIkFXU19MQU1CREFfRlVOQ1RJT05fTUVNT1JZX1NJWkUiOiAiMTI4IiwKICAiTk9ERV9FWFRSQV9DQV9DRVJUUyI6ICIvZXRjL3BraS90bHMvY2VydHMvY2EtYnVuZGxlLmNydCIsCiAgIl9YX0FNWk5fVFJBQ0VfSUQiOiAiUm9vdD0xLTYzMjc1OWZlLTA3MTdkNTc0MTVhM2NlNjMzNjljN2I0MztQYXJlbnQ9MTM4NGI3NzMxNmU3MTcyZDtTYW1wbGVkPTAiCn0KMjAyMi0wOS0xOFQxNzo0ODo0Ni4wOTdaCWZiYzEwYmYwLTJlZmQtNGRhNy05MDRjLWE0YTgxMDJjMDUzMwlJTkZPCUVWRU5UCnt9CkVORCBSZXF1ZXN0SWQ6IGZiYzEwYmYwLTJlZmQtNGRhNy05MDRjLWE0YTgxMDJjMDUzMwpSRVBPUlQgUmVxdWVzdElkOiBmYmMxMGJmMC0yZWZkLTRkYTctOTA0Yy1hNGE4MTAyYzA1MzMJRHVyYXRpb246IDEuNDggbXMJQmlsbGVkIER1cmF0aW9uOiAyIG1zCU1lbW9yeSBTaXplOiAxMjggTUIJTWF4IE1lbW9yeSBVc2VkOiA1NyBNQgkK",
    "ExecutedVersion": "$LATEST"
}

Or:

aws lambda invoke \
  --function-name my-test-lambda-nodejs-function
  out --log-type Tail | jq .LogResult -r | base64 --decode

START RequestId: 6c293d94-6af9-4217-9ed4-ad0abb28870f Version: $LATEST
2022-09-18T17:49:20.143Z    6c293d94-6af9-4217-9ed4-ad0abb28870f    INFO    ENVIRONMENT VARIABLES
{
  "AWS_LAMBDA_FUNCTION_VERSION": "$LATEST",
  "AWS_SESSION_TOKEN": "IQoJb3JpZ2luX2VjEIr//////////wEaCXVzLWVhc3QtMSJHMEUCICdBey91XrsoN7V6khFH0NJlUxnW/i7Cvp9F6+BKCDR9AiEAsY7UC3fquezpjowBk0DNSMIDcjSyv7cTT2BXTXBn6jgqkQMIMxADGgw0NjIwOTI3ODA0NjYiDAsaqepN5dnQoCyjISruAlEmARHE5sVJJY7dpcf/eaqwwLfC1G8rbv8wdcntu/YQUgZ0+xkKlfQHopSRZWsaCxrhPjF+BT4oPh9cLXSdvdXTKeU8c5sI66cVUKsOwkKcW+jh6jzLzoAzIvArbPxown/pfH0ACgq4alSDUAgrUbbWwR6MUoQl9HLRgvVqDZ9Rntz9zNpw0SJgKIRY4XfZ1xSPLigTdfnTKq5ckTAhQCXH+lSxYmviilDbMQwec2Rs6/w3IgB92AIp2XvEM/M0Fvn+MPn4B149CjbhTBNeFvQeZF7ZgxoT5yaoxTK8aA/F2LofBKxG1tjLrx799qpCBVggxnAOqQ+OWIMAdV6ca1d5AmRnrlwHIC3Wqd/QpLcAM7S/zoIN5FNUBmV9yLpnEGjUURKaMEkNqfWu0UoeVhhuOcXpuVRGMsdCS1tujg6BbXqrVKE9+NndwwQUQkHwGKJpiNkeWYb2b2D6ZxZfyTjJ0tjsmcdffq9M/NCFRjD6s52ZBjqdAXGUUNpZmaPIc/6N2gmA5feLzrfivPHZTC4+uj026TSKEBlkdOM977dOo0LLbO/x+ue3EvNY6UkSuzOdBKQ5en52pUpPRgzE5oYB5XYG1RYsLZVLzv/e73fhtNzkoTZdtgb+aRYGWo73dz1RC76ru59ovP0HOX6E5mtmNmeZxQSPoD5PjdqS8XAd9+Tp00ADggr5w3S92AVTQj/dgpw=",
  "AWS_LAMBDA_LOG_GROUP_NAME": "/aws/lambda/my-test-lambda-nodejs-function",
  "LD_LIBRARY_PATH": "/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib",
  "LAMBDA_TASK_ROOT": "/var/task",
  "AWS_LAMBDA_RUNTIME_API": "127.0.0.1:9001",
  "AWS_LAMBDA_LOG_STREAM_NAME": "2022/09/18/[$LATEST]73deb74dd8b547378d7418d025dda670",
  "AWS_EXECUTION_ENV": "AWS_Lambda_nodejs16.x",
  "AWS_LAMBDA_FUNCTION_NAME": "my-test-lambda-nodejs-function",
  "AWS_XRAY_DAEMON_ADDRESS": "169.254.79.129:2000",
  "PATH": "/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin",
  "AWS_DEFAULT_REGION": "us-east-1",
  "PWD": "/var/task",
  "AWS_SECRET_ACCESS_KEY": "PW9tkAiblScAriPmxhtMUikYQoRkKD7FvkKHnitj",
  "LAMBDA_RUNTIME_DIR": "/var/runtime",
  "LANG": "en_US.UTF-8",
  "AWS_LAMBDA_INITIALIZATION_TYPE": "on-demand",
  "NODE_PATH": "/opt/nodejs/node16/node_modules:/opt/nodejs/node_modules:/var/runtime/node_modules:/var/runtime:/var/task",
  "TZ": ":UTC",
  "AWS_REGION": "us-east-1",
  "AWS_ACCESS_KEY_ID": "ASIAWXFXAIOZMYTT6AMG",
  "SHLVL": "0",
  "_AWS_XRAY_DAEMON_ADDRESS": "169.254.79.129",
  "_AWS_XRAY_DAEMON_PORT": "2000",
  "AWS_XRAY_CONTEXT_MISSING": "LOG_ERROR",
  "_HANDLER": "index.handler",
  "AWS_LAMBDA_FUNCTION_MEMORY_SIZE": "128",
  "NODE_EXTRA_CA_CERTS": "/etc/pki/tls/certs/ca-bundle.crt",
  "_X_AMZN_TRACE_ID": "Root=1-63275a20-5e0ae45d25fd0b166aa8ced5;Parent=3a5d154b313dd6df;Sampled=0"
}
2022-09-18T17:49:20.143Z    6c293d94-6af9-4217-9ed4-ad0abb28870f    INFO    EVENT
{}
END RequestId: 6c293d94-6af9-4217-9ed4-ad0abb28870f
REPORT RequestId: 6c293d94-6af9-4217-9ed4-ad0abb28870f  Duration: 10.98 ms  Billed Duration: 11 ms  Memory Size: 128 MB Max Memory Used: 57 MB  

[simonw]

I had to upgrade awscli first. I did that using:

/tmp % head -n 1 $(which aws) 
#!/usr/local/opt/python@3.9/bin/python3.9

Then the upgrade with:

/usr/local/opt/python@3.9/bin/pip3 install -U awscli

Then I ran this:

aws lambda create-function-url-config \
  --function-name my-test-lambda-nodejs-function \
  --auth-type NONE

{
    "FunctionUrl": "https://wtqdq25qylflndenvxzlssyhmi0zksbf.lambda-url.us-east-1.on.aws/",
    "FunctionArn": "arn:aws:lambda:us-east-1:462092780466:function:my-test-lambda-nodejs-function",
    "AuthType": "NONE",
    "CreationTime": "2022-09-18T17:55:55.359524Z"
}

[simonw]

https://wtqdq25qylflndenvxzlssyhmi0zksbf.lambda-url.us-east-1.on.aws/ returns:

{
  "Message": "Forbidden"
}

Probably because I need to deploy a function that actually handles HTTP requests. That's the next step - I'll likely switch to Python at this point too.

[simonw]

Generated this with GPT-3:

/* A node.js AWS lambda function that returns a 200 text/html response saying "HEllo World" */

exports.handler = async function(event, context) {
    return {
        statusCode: 200,
        headers: {
            "Content-Type": "text/html"
        },
        body: "<h1>Hello World</h1>"
    };
};

[simonw]

rm -f function.zip
zip function.zip index.js

aws lambda update-function-code \
  --function-name my-test-lambda-nodejs-function \
  --zip-file fileb://function.zip

[simonw]

Still getting that Forbidden error from https://wtqdq25qylflndenvxzlssyhmi0zksbf.lambda-url.us-east-1.on.aws/ - spotted this HTTP header:

x-amzn-ErrorType: AccessDeniedException

I thought this would fix that:

aws lambda create-function-url-config \
  --function-name my-test-lambda-nodejs-function \
  --auth-type NONE

[simonw]

Looking at https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#

[simonw]

https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/functions/my-test-lambda-nodejs-function?tab=monitoring has logs.

[simonw]

On https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/functions/my-test-lambda-nodejs-function?tab=configure found this:

To allow unauthenticated requests, choose the Permissions tab and and create a resource-based policy that grants lambda:invokeFunctionUrl permissions to all principals (*).

[simonw]

Another tutorial: https://docs.aws.amazon.com/lambda/latest/dg/urls-tutorial.html

That tutorial includes this step (I added my-test-lambda-nodejs-function here):

aws lambda add-permission \
  --function-name my-test-lambda-nodejs-function \
  --action lambda:InvokeFunctionUrl \
  --principal "*" \
  --function-url-auth-type "NONE" \
  --statement-id url

Response:

{
    "Statement": "{\"Sid\":\"url\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"lambda:InvokeFunctionUrl\",\"Resource\":\"arn:aws:lambda:us-east-1:462092780466:function:my-test-lambda-nodejs-function\",\"Condition\":{\"StringEquals\":{\"lambda:FunctionUrlAuthType\":\"NONE\"}}}"
}

Decoded:

/tmp % echo '{
    "Statement": "{\"Sid\":\"url\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"lambda:InvokeFunctionUrl\",\"Resource\":\"arn:aws:lambda:us-east-1:462092780466:function:my-test-lambda-nodejs-function\",\"Condition\":{\"StringEquals\":{\"lambda:FunctionUrlAuthType\":\"NONE\"}}}"
}' | jq .Statement -r | jq

{
  "Sid": "url",
  "Effect": "Allow",
  "Principal": "*",
  "Action": "lambda:InvokeFunctionUrl",
  "Resource": "arn:aws:lambda:us-east-1:462092780466:function:my-test-lambda-nodejs-function",
  "Condition": {
    "StringEquals": {
      "lambda:FunctionUrlAuthType": "NONE"
    }
  }
}

And now this works! https://wtqdq25qylflndenvxzlssyhmi0zksbf.lambda-url.us-east-1.on.aws/

[simonw]

OK let's try a Python one instead. I'll call that my-test-lambda-nodejs-function.

Trying this as lambda_function.py:

def lambda_handler(event, context): 
    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "text/html"
        },
        "body": "<h1>Hello World</h1>"
    }

rm -f function.zip 
zip function.zip lambda_function.py 

[simonw]

aws lambda create-function \
  --function-name my-test-lambda-python-function \
  --zip-file fileb://function.zip \
  --runtime python3.9 \
  --handler "lambda_function.lambda_handler" \
  --role "arn:aws:iam::${AWS_ACCOUNT_ID}:role/lambda-ex"

Reply:

{
    "FunctionName": "my-test-lambda-python-function",
    "FunctionArn": "arn:aws:lambda:us-east-1:462092780466:function:my-test-lambda-python-function",
    "Runtime": "python3.9",
    "Role": "arn:aws:iam::462092780466:role/lambda-ex",
    "Handler": "lambda_function.lambda_handler",
    "CodeSize": 323,
    "Description": "",
    "Timeout": 3,
    "MemorySize": 128,
    "LastModified": "2022-09-18T19:11:15.900+0000",
    "CodeSha256": "TfPlhn/OBnr7Pn3zb3rm8FTMKWUD92XG1YsRGfvkjvU=",
    "Version": "$LATEST",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "afb3a3e0-5187-4a2b-ab52-a2e0232f27c5",
    "State": "Pending",
    "StateReason": "The function is being created.",
    "StateReasonCode": "Creating",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ],
    "EphemeralStorage": {
        "Size": 512
    }
}

I tried removing --handler index.handler because I'm using the lambda_function.lambda_handler default value instead, but I got this error:

An error occurred (InvalidParameterValueException) when calling the CreateFunction operation: Runtime and Handler are mandatory parameters for functions created with deployment packages.

I got python3.9 from aws lambda create-function help.

[simonw]

Next:

aws lambda add-permission \
  --function-name my-test-lambda-python-function \
  --action lambda:InvokeFunctionUrl \
  --principal "*" \
  --function-url-auth-type "NONE" \
  --statement-id url

aws lambda create-function-url-config \
  --function-name my-test-lambda-python-function \
  --auth-type NONE

https://fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws/ seems to work now - but it's giving me the same response as the JavaScript one did.

[simonw]

Update that to:

def lambda_handler(event, context): 
    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "text/html"
        },
        "body": "<h1>Hello World from Python</h1>"
    }

Then re-deploy with:

rm -f function.zip 
zip function.zip lambda_function.py 
aws lambda update-function-code \
  --function-name my-test-lambda-python-function \
  --zip-file fileb://function.zip

That worked!

https://fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws/ now returns "Hello World from Python".

[simonw]

OK, I'm going to try deploying Datasette(!).

[simonw]

OMG that worked! https://fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws/

build-function-zip.sh:

#!/bin/bash
python3 -m pip install -t lib -r requirements.txt

rm -f lambda_function.zip

(cd lib; zip ../lambda_function.zip -r .)

# Now add my code
zip lambda_function.zip lambda_function.py

Then deploy like this:

aws lambda update-function-code \
 --function-name my-test-lambda-python-function \
 --zip-file fileb://lambda_function.zip

[simonw]

First hit gets this error:

Second hit got it too. Third hit worked fine.

[simonw]

I bundled fixtures.db by doing this:

wget latest.datasette.io/fixtures.db
zip lambda_function.zip fixtures.db

Then modified my lambda_function.py to this:

import mangum
from datasette.app import Datasette

ds = Datasette(["fixtures.db"])

lambda_handler = mangum.Mangum(ds.app())

Then this to over-write the zipped version of that file with the new one:

zip lambda_function.zip lambda_function.py

Then deploy:

aws lambda update-function-code \
 --function-name my-test-lambda-python-function \
 --zip-file fileb://lambda_function.zip

And now: https://fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws/fixtures/sortable

[simonw]

Note that I'm not calling await ds.invoke_startup() yet.

[simonw]

Mangum class constructor is useful: https://github.com/jordaneremieff/mangum/blob/c58e85745d6bd7c9cf8734398750e179751b716d/mangum/adapter.py#L30-L47

It tries these handlers in turn:

HANDLERS: List[Type[LambdaHandler]] = [
    ALB,
    HTTPGateway,
    APIGateway,
    LambdaAtEdge,
]

[simonw]

It looks like it sets the original AWS event and context inside the ASGI scope: https://github.com/jordaneremieff/mangum/blob/c58e85745d6bd7c9cf8734398750e179751b716d/mangum/handlers/api_gateway.py#L195-L196

            "aws.event": self.event,
            "aws.context": self.context,

[simonw]

I'm going to build datasette-debug-mangum inspired by https://datasette.io/plugins/datasette-debug-asgi

[simonw]

Actually I don't think I need that, I can use datasette-debug-asgi directly to see what's in those scope variables.

[simonw]

Potential problem: if a plugin is installed in lib/ will that be enough for Datasette to detect that plugin and use it?

[simonw]

No plugins worked fine! I added datasette-debug-asgi to the requirements.txt and re-ran the build and everything was fine.

https://fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws/-/asgi-scope

{'actor': None,
 'asgi': {'spec_version': '2.0', 'version': '3.0'},
 'aws.context': LambdaContext([aws_request_id=ed80a0c9-244c-4b75-82e4-af986847ed85,log_group_name=/aws/lambda/my-test-lambda-python-function,log_stream_name=2022/09/18/[$LATEST]58f6220eca614d88bcaff45aadc6d4cc,function_name=my-test-lambda-python-function,memory_limit_in_mb=128,function_version=$LATEST,invoked_function_arn=arn:aws:lambda:us-east-1:462092780466:function:my-test-lambda-python-function,client_context=None,identity=CognitoIdentity([cognito_identity_id=None,cognito_identity_pool_id=None])]),
 'aws.event': {'headers': {'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
                           'accept-encoding': 'gzip, deflate, br',
                           'accept-language': 'en-US,en;q=0.5',
                           'dnt': '1',
                           'host': 'fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws',
                           'sec-fetch-dest': 'document',
                           'sec-fetch-mode': 'navigate',
                           'sec-fetch-site': 'none',
                           'sec-fetch-user': '?1',
                           'upgrade-insecure-requests': '1',
                           'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS '
                                         'X 10.15; rv:104.0) Gecko/20100101 '
                                         'Firefox/104.0',
                           'x-amzn-tls-cipher-suite': 'ECDHE-RSA-AES128-GCM-SHA256',
                           'x-amzn-tls-version': 'TLSv1.2',
                           'x-amzn-trace-id': 'Root=1-632776fb-1e23c57114bc2cdd086054fb',
                           'x-forwarded-for': '24.5.172.176',
                           'x-forwarded-port': '443',
                           'x-forwarded-proto': 'https'},
               'isBase64Encoded': False,
               'rawPath': '/-/asgi-scope',
               'rawQueryString': '',
               'requestContext': {'accountId': 'anonymous',
                                  'apiId': 'fnkvspusjrl5dxytaxnuwidxem0hverw',
                                  'domainName': 'fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws',
                                  'domainPrefix': 'fnkvspusjrl5dxytaxnuwidxem0hverw',
                                  'http': {'method': 'GET',
                                           'path': '/-/asgi-scope',
                                           'protocol': 'HTTP/1.1',
                                           'sourceIp': '24.5.172.176',
                                           'userAgent': 'Mozilla/5.0 '
                                                        '(Macintosh; Intel Mac '
                                                        'OS X 10.15; rv:104.0) '
                                                        'Gecko/20100101 '
                                                        'Firefox/104.0'},
                                  'requestId': 'ed80a0c9-244c-4b75-82e4-af986847ed85',
                                  'routeKey': '$default',
                                  'stage': '$default',
                                  'time': '18/Sep/2022:19:52:27 +0000',
                                  'timeEpoch': 1663530747090},
               'routeKey': '$default',
               'version': '2.0'},
 'client': ('24.5.172.176', 0),
 'csrftoken': <function asgi_csrf_decorator.<locals>._asgi_csrf_decorator.<locals>.app_wrapped_with_csrf.<locals>.get_csrftoken at 0x7f41e52aa550>,
 'headers': [[b'sec-fetch-mode', b'navigate'],
             [b'x-amzn-tls-version', b'TLSv1.2'],
             [b'sec-fetch-site', b'none'],
             [b'accept-language', b'en-US,en;q=0.5'],
             [b'x-forwarded-proto', b'https'],
             [b'x-forwarded-port', b'443'],
             [b'dnt', b'1'],
             [b'x-forwarded-for', b'24.5.172.176'],
             [b'sec-fetch-user', b'?1'],
             [b'accept',
              b'text/html,application/xhtml+xml,application/xml;q=0.9,image/'
              b'avif,image/webp,*/*;q=0.8'],
             [b'x-amzn-tls-cipher-suite', b'ECDHE-RSA-AES128-GCM-SHA256'],
             [b'x-amzn-trace-id', b'Root=1-632776fb-1e23c57114bc2cdd086054fb'],
             [b'host',
              b'fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws'],
             [b'upgrade-insecure-requests', b'1'],
             [b'accept-encoding', b'gzip, deflate, br'],
             [b'sec-fetch-dest', b'document'],
             [b'user-agent',
              b'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:104.0) Geck'
              b'o/20100101 Firefox/104.0']],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/-/asgi-scope',
 'query_string': b'',
 'raw_path': None,
 'root_path': '',
 'scheme': 'https',
 'server': ('fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws',
            443),
 'type': 'http',
 'url_route': {'kwargs': {}}}

[simonw]

Updated my build-function-zip.sh script to this:

#!/bin/bash
python3 -m pip install -t lib -r requirements.txt

rm -f lambda_function.zip

(cd lib; zip ../lambda_function.zip -r .)

# Now add my code
zip lambda_function.zip lambda_function.py

# And my database
zip lambda_function.zip fixtures.db

[simonw]

I'm going to add datasette-graphql just to see what happens.

[simonw]

Getting this intermittent error:

[simonw]

Found this in the error logs: https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/$252Faws$252Flambda$252Fmy-test-lambda-python-function/log-events/2022$252F09$252F18$252F$255B$2524LATEST$255D38c93bce94a3490393176be896db6b14$3Fstart$3DPT3H

Looks like the error is happening in populate_schema_tables().

[simonw]

Managed to get https://fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws/-/versions to load after a few refreshes:

{
    "python": {
        "version": "3.9.13",
        "full": "3.9.13 (main, Jun 10 2022, 16:49:31) \n[GCC 7.3.1 20180712 (Red Hat 7.3.1-15)]"
    },
    "datasette": {
        "version": "0.62"
    },
    "asgi": "3.0",
    "uvicorn": "0.18.3",
    "sqlite": {
        "version": "3.7.17",
        "fts_versions": [
            "FTS4",
            "FTS3"
        ],
        "extensions": {},
        "compile_options": [
            "DISABLE_DIRSYNC",
            "ENABLE_COLUMN_METADATA",
            "ENABLE_FTS3",
            "ENABLE_RTREE",
            "ENABLE_UNLOCK_NOTIFY",
            "SECURE_DELETE",
            "TEMP_STORE=1",
            "THREADSAFE=1"
        ]
    }
}

That is an ANCIENT SQLite version. 3.7.17 is from 2013-05-20.

[simonw]

datasette-publish-vercel solves that with pysqlite3-binary.

[simonw]

Tried adding pysqlite3-binary to my requirements.txt but macOS doesn't know what to do with that:

ERROR: Could not find a version that satisfies the requirement pysqlite3-binary (from versions: none)
ERROR: No matching distribution found for pysqlite3-binary

So I need to run a build of the zip file on an Amazon Linux machine I think.

[simonw]

I bet I can run a Docker image that can compile that zip file for me - one that produces a .zip from a list of requriments.txt would be AMAZING.

[simonw]

FROM amazonlinux:2
RUN yum install -y python3 python3-pip
RUN python3 -m pip install boto3 pandas s3fs
COPY src /code
ENTRYPOINT [ "python3", "/code/main.py" ]

From https://stackoverflow.com/a/61377103/6083

[simonw]

https://github.com/aws-samples/aws-cost-explorer-report is a good example of something that uses Docker to build a Lambda function zip.

[simonw]

Another clue in https://github.com/aws-samples/aws-lambda-layer-create-script

#!/bin/bash

if [ "$1" != "" ] || [$# -gt 1]; then
    echo "Creating layer compatible with python version $1"
    docker run -v "$PWD":/var/task "lambci/lambda:build-python$1" /bin/sh -c "pip install -r requirements.txt -t python/lib/python$1/site-packages/; exit"
    zip -r layer.zip python > /dev/null
    rm -r python
    echo "Done creating layer!"
    ls -lah layer.zip

else
    echo "Enter python version as argument - ./createlayer.sh 3.6"
fi

[simonw]

This issue:

• https://github.com/lambci/docker-lambda/issues/343 Seems to suggest that's unmaintained, should use this instead: public.ecr.aws/sam/build-python3.9 - https://gallery.ecr.aws/sam/build-python3.9

Docs: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-image-repositories.html

[simonw]

Useful search results: https://cs.github.com/?scopeName=All+repos&scope=&q=docker+public.ecr.aws%2Fsam%2Fbuild-python3.9

[simonw]

I like the look of https://github.com/hmrc/grant-ssh-access/blob/b7515aa1933723855c269c381f97794fbcdc7738/Jenkinsfile#L20

        sh('docker run -t -v $(pwd):/data public.ecr.aws/sam/build-python3.9:latest  /data/package.sh')

Where package.sh contains:

!/bin/bash

set -xeou

BASEDIR=/data
PIPPACKAGESDIR=${BASEDIR}/lambda-packages

cd ${BASEDIR}

zip grant_ssh_access.zip grant_ssh_access.py mdtp.pem

mkdir -p ${PIPPACKAGESDIR}
pip install -t ${PIPPACKAGESDIR} -r requirements.txt
cd ${PIPPACKAGESDIR}
zip -r ../grant_ssh_access.zip .
rm -rf ${PIPPACKAGESDIR}/*

[simonw]

So could I do this?

docker run -t -v $(pwd):/mnt public.ecr.aws/sam/build-python3.9:latest \
  /bin/sh -c "pip install -r /mnt/requirements.txt -t /mnt/lib"

Trying that now - I deleted my lib folder first.

[simonw]

That might have worked! lib/ is populated with the expected files.

[simonw]

Ran this:

rm -f lambda_function.zip
(cd lib; zip ../lambda_function.zip -r .)
zip lambda_function.zip lambda_function.py
zip lambda_function.zip fixtures.db

And ./update.sh to update the deployed function.

[simonw]

https://fnkvspusjrl5dxytaxnuwidxem0hverw.lambda-url.us-east-1.on.aws/-/versions now shows:

{
    "python": {
        "version": "3.9.13",
        "full": "3.9.13 (main, Jun 10 2022, 16:49:31) \n[GCC 7.3.1 20180712 (Red Hat 7.3.1-15)]"
    },
    "datasette": {
        "version": "0.62"
    },
    "asgi": "3.0",
    "uvicorn": "0.18.3",
    "sqlite": {
        "version": "3.39.2",
        "fts_versions": [
            "FTS5",
            "FTS4",
            "FTS3"
        ],
        "extensions": {
            "json1": null
        },
        "compile_options": [
            "ATOMIC_INTRINSICS=1",
            "COMPILER=gcc-6.3.0 20170516",
            "DEFAULT_AUTOVACUUM",
            "DEFAULT_CACHE_SIZE=-2000",
            "DEFAULT_FILE_FORMAT=4",
            "DEFAULT_JOURNAL_SIZE_LIMIT=-1",
            "DEFAULT_MMAP_SIZE=0",
            "DEFAULT_PAGE_SIZE=4096",
            "DEFAULT_PCACHE_INITSZ=20",
            "DEFAULT_RECURSIVE_TRIGGERS",
            "DEFAULT_SECTOR_SIZE=4096",
            "DEFAULT_SYNCHRONOUS=2",
            "DEFAULT_WAL_AUTOCHECKPOINT=1000",
            "DEFAULT_WAL_SYNCHRONOUS=2",
            "DEFAULT_WORKER_THREADS=0",
            "ENABLE_FTS3",
            "ENABLE_FTS3_PARENTHESIS",
            "ENABLE_FTS4",
            "ENABLE_FTS5",
            "ENABLE_LOAD_EXTENSION",
            "ENABLE_MATH_FUNCTIONS",
            "ENABLE_RTREE",
            "ENABLE_STAT4",
            "ENABLE_UPDATE_DELETE_LIMIT",
            "MALLOC_SOFT_LIMIT=1024",
            "MAX_ATTACHED=10",
            "MAX_COLUMN=2000",
            "MAX_COMPOUND_SELECT=500",
            "MAX_DEFAULT_PAGE_SIZE=8192",
            "MAX_EXPR_DEPTH=1000",
            "MAX_FUNCTION_ARG=127",
            "MAX_LENGTH=1000000000",
            "MAX_LIKE_PATTERN_LENGTH=50000",
            "MAX_MMAP_SIZE=1099511627776",
            "MAX_PAGE_COUNT=1073741823",
            "MAX_PAGE_SIZE=65536",
            "MAX_SQL_LENGTH=1000000000",
            "MAX_TRIGGER_DEPTH=1000",
            "MAX_VARIABLE_NUMBER=250000",
            "MAX_VDBE_OP=250000000",
            "MAX_WORKER_THREADS=0",
            "MUTEX_PTHREADS",
            "SOUNDEX",
            "SYSTEM_MALLOC",
            "TEMP_STORE=3",
            "THREADSAFE=1",
            "USE_URI"
        ]
    },
    "pysqlite3": "0.4.7.post7"
}

And I'm not getting that parameter error any more, which I think was caused by the old SQLite version.