Postgres migrations support

[Rubyj]

@koxudaxi

Do you think this plug-in could be used to support database migrations using alembic/sqlalchemy?

[koxudaxi]

Thank you for creating this issue. The answer was No. because the client converts requests and responses to boto3.

I thought about your question. I feel that people want to use the client as SQL Alchemy driver. I have written the feature for fully supporting SQL Alchemy. https://github.com/koxudaxi/py-data-api/pull/20 The feature is an experimental phase, and I try only execute a simple query like select * from table However, You may use the migration library with this client.

an example:

from sqlalchemy.engine import create_engine
engine = create_engine(
        'mysql+pydataapi://',
        connect_args={
            'resource_arn': 'arn:aws:rds:us-east-1:123456789012:cluster:dummy',
            'secret_arn': 'arn:aws:secretsmanager:us-east-1:123456789012:secret:dummy',
            'database': 'test'}
)
result = engine.execute("select * from pets")
print(result.fetchall())
# [(1, 'dog'),
# (2, 'cat'),
# (3, 'snake')]

[Rubyj]

@koxudaxi So, I could, in theory, use this SQLAlchemy connection with alembic?

[koxudaxi]

So, I could, in theory, use this SQLAlchemy connection with alembic? Also, what is secret ARN here being used for?

Yes. However, I think we need a lot of tests.

Also, what is secret ARN here being used for?

We use resource_arn and secret_arn to connect aurora DataAPI. The values are dummy ARNs for local-data-api. I develop DataAPI Mock server for a local machine The server run by docker-compose in our local machine. Also, we set dummy ARNs as the environment variables on the server.

[Rubyj]

@koxudaxi will this work for postgres aurora? Your examples only seem to work for mysql.

When I try engine = create_engine('postgres+pydataapi://', connect_args=args)

I get an error

NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres.pydataapi

[koxudaxi]

Would you please try postgresql+pydataapi:// ?

[Rubyj]

That worked! thank you!

[Rubyj]

Hi @koxudaxi I have tried to use this with migrations using the working url and I run into the following error. Although, it looks like it almost works. Any ideas?

Should I add this param to the PostgreSQLDataAPIDialect class

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 48393aac95df, create variant table
Traceback (most recent call last):
  File "/home/rjacobs/git/vcfparserlambda/venv/bin/alembic", line 10, in <module>
    sys.exit(main())
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/config.py", line 573, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/config.py", line 567, in main
    self.run_cmd(cfg, options)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/config.py", line 547, in run_cmd
    **dict((k, getattr(options, k, None)) for k in kwarg)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
    script.run_env()
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/util/compat.py", line 173, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "alembic/env.py", line 79, in <module>
    run_migrations_online()
  File "alembic/env.py", line 73, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/runtime/migration.py", line 518, in run_migrations
    step.migration_fn(**kw)
  File "/home/rjacobs/git/vcfparserlambda/alembic/versions/48393aac95df_create_variant_table.py", line 22, in upgrade
    sa.Column('id', sa.Integer, primary_key=True)
  File "<string>", line 8, in create_table
  File "<string>", line 3, in create_table
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/operations/ops.py", line 1250, in create_table
    return operations.invoke(op)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/operations/base.py", line 345, in invoke
    return fn(self, operation)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/operations/toimpl.py", line 101, in create_table
    operations.impl.create_table(table)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/ddl/impl.py", line 252, in create_table
    self._exec(schema.CreateTable(table))
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/ddl/impl.py", line 134, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1043, in _execute_ddl
    else None,
  File "<string>", line 1, in <lambda>
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 462, in compile
    return self._compiler(dialect, bind=bind, **kw)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 29, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 319, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 350, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 92, in _compiler_dispatch
    return meth(self, **kw)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 2860, in visit_create_table
    create_column, first_pk=column.primary_key and not first_pk
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 350, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 92, in _compiler_dispatch
    return meth(self, **kw)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/compiler.py", line 2892, in visit_create_column
    text = self.get_column_specification(column, first_pk=first_pk)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/base.py", line 1851, in get_column_specification
    self.dialect.supports_smallserial
AttributeError: 'PostgreSQLDataAPIDialect' object has no attribute 'supports_smallserial'

[Rubyj]

I solved the above issue by adding the supports_smallserial = True field to the PostgreSQLDataAPIDialect class.

Now I receive the following error

Traceback (most recent call last):
  File "/home/rjacobs/git/vcfparserlambda/venv/bin/alembic", line 10, in <module>
    sys.exit(main())
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/config.py", line 573, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/config.py", line 567, in main
    self.run_cmd(cfg, options)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/config.py", line 547, in run_cmd
    **dict((k, getattr(options, k, None)) for k in kwarg)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
    script.run_env()
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/util/compat.py", line 173, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "alembic/env.py", line 79, in <module>
    run_migrations_online()
  File "alembic/env.py", line 73, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/runtime/migration.py", line 525, in run_migrations
    head_maintainer.update_to_step(step)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/runtime/migration.py", line 716, in update_to_step
    self._insert_version(vers)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/runtime/migration.py", line 666, in _insert_version
    version_num=literal_column("'%s'" % version)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/alembic/ddl/impl.py", line 134, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
    distilled_params,
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1265, in _execute_context
    result = context._setup_crud_result_proxy()
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 1118, in _setup_crud_result_proxy
    self._setup_ins_pk_from_lastrowid()
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 1154, in _setup_ins_pk_from_lastrowid
    lastrowid = self.get_lastrowid()
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 1089, in get_lastrowid
    return self.cursor.lastrowid
AttributeError: 'Cursor' object has no attribute 'lastrowid'

[koxudaxi]

OK, Just a minute.

[koxudaxi]

I guess we should change DataAPIDialect to PGDialect

class PostgreSQLDataAPIDialect(PGDialect):

[Rubyj]

Do we need the PostgreSQLDataAPIDialect to extend from both PGDialect and DataAPIDialect?

[koxudaxi]

no only PGDialect I'm testing this now

class PostgreSQLDataAPIDialect(PGDialect):
    def get_primary_keys(self, connection, table_name, schema=None, **kw):
        pass

    def get_temp_table_names(self, connection, schema=None, **kw):
        pass

    def get_temp_view_names(self, connection, schema=None, **kw):
        pass

    def normalize_name(self, name):
        return name

    def denormalize_name(self, name):
        return name

    @classmethod
    def dbapi(cls) -> Type[Connection]:
        return Connection

[Rubyj]

@koxudaxi That works! Form my testing, so does:

class PostgreSQLDataAPIDialect(PGDialect, DataAPIDialect):
    name = "postgresql"
    supports_alter = True
    max_identifier_length = 63
    supports_sane_rowcount = True
    statement_compiler = PGCompiler
    ddl_compiler = PGDDLCompiler
    type_compiler = PGTypeCompiler
    preparer = PGIdentifierPreparer
    inspector = PGInspector
    isolation_level = None

[koxudaxi]

Great work!! Thank you 😄

[Rubyj]

Which would you prefer? I can fork the code and submit a PR.

[koxudaxi]

Would you please create a PR?

[Rubyj]

Yes! I can do that. Would you like the solution you posted or the one I posted?

[koxudaxi]

I choise your solution.

[koxudaxi]

I have released this package include the feature. version is 0.4.1 You help to improve the package 👍 Thank you!!

[Rubyj]

@koxudaxi After pulling new version of this library, I seem to now have an issue when trying to undo migrations I.E. drop a row from a table. The query continues to return no results and I get the following error from sqlalchemy/alembic:

ERROR [alembic.util.messaging] Online migration expected to match one row when deleting 'test_revision' in 'alembic_version'; 0 found
  FAILED: Online migration expected to match one row when deleting 'test_revision' in 'alembic_version'; 0 found

I checked the database and this row is there, but for some reason the query is not returning the results.

[koxudaxi]

Interesting.

After pulling new version of this library,

Is it the same result? when did you use it with the old version?

Also, Could you show me queries? if you can set echo=True then, it shows me queries.

[Rubyj]

Very interesting. I tried new version and old version with our changes. Same error occurs. Where do I set echo=True? In the connection string?

I do not remember this error happening on old version with our changes, but it seems to be happening now even on old or new with our changes.

[koxudaxi]

I have changed only the reset transaction. The library clear transaction and begin a new transaction after the library call commit or rollback. However, I guess It is not a problem.

The cookbook helps you to use echo. https://alembic.sqlalchemy.org/en/latest/cookbook.html?highlight=echo

[Rubyj]

Here is the echo log for undoing a migrations (should delete a table).

2019-10-14 13:04:28,763 INFO sqlalchemy.engine.base.Engine select version()
INFO  [sqlalchemy.engine.base.Engine] select version()
2019-10-14 13:04:28,763 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
2019-10-14 13:04:28,791 INFO sqlalchemy.engine.base.Engine select current_schema()
INFO  [sqlalchemy.engine.base.Engine] select current_schema()
2019-10-14 13:04:28,791 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
2019-10-14 13:04:28,850 INFO sqlalchemy.engine.base.Engine SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
INFO  [sqlalchemy.engine.base.Engine] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
2019-10-14 13:04:28,850 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
2019-10-14 13:04:28,882 INFO sqlalchemy.engine.base.Engine SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
INFO  [sqlalchemy.engine.base.Engine] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
2019-10-14 13:04:28,882 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
2019-10-14 13:04:28,914 INFO sqlalchemy.engine.base.Engine show standard_conforming_strings
INFO  [sqlalchemy.engine.base.Engine] show standard_conforming_strings
2019-10-14 13:04:28,914 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
2019-10-14 13:04:28,984 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
INFO  [sqlalchemy.engine.base.Engine] BEGIN (implicit)
2019-10-14 13:04:29,011 INFO sqlalchemy.engine.base.Engine select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=:name
INFO  [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=:name
2019-10-14 13:04:29,012 INFO sqlalchemy.engine.base.Engine {'name': 'alembic_version'}
INFO  [sqlalchemy.engine.base.Engine] {'name': 'alembic_version'}
2019-10-14 13:04:29,038 INFO sqlalchemy.engine.base.Engine SELECT alembic_version.version_num 
FROM alembic_version
INFO  [sqlalchemy.engine.base.Engine] SELECT alembic_version.version_num 
FROM alembic_version
2019-10-14 13:04:29,039 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [alembic.runtime.migration] Running downgrade test_revision -> , create variant table
2019-10-14 13:04:29,066 INFO sqlalchemy.engine.base.Engine 
DROP TABLE variant
INFO  [sqlalchemy.engine.base.Engine] 
DROP TABLE variant
2019-10-14 13:04:29,066 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
2019-10-14 13:04:29,096 INFO sqlalchemy.engine.base.Engine DELETE FROM alembic_version WHERE alembic_version.version_num = 'test_revision'
INFO  [sqlalchemy.engine.base.Engine] DELETE FROM alembic_version WHERE alembic_version.version_num = 'test_revision'
2019-10-14 13:04:29,097 INFO sqlalchemy.engine.base.Engine {}
INFO  [sqlalchemy.engine.base.Engine] {}
2019-10-14 13:04:29,128 INFO sqlalchemy.engine.base.Engine ROLLBACK
INFO  [sqlalchemy.engine.base.Engine] ROLLBACK
ERROR [alembic.util.messaging] Online migration expected to match one row when deleting 'test_revision' in 'alembic_version'; 0 found
  FAILED: Online migration expected to match one row when deleting 'test_revision' in 'alembic_version'; 0 found

@koxudaxi I noticed that on version 0.4.0 with our PGDialect change, I still get this error, but the deletion does work. On the newest version, 0.4.2 or even 0.4.1, I also get this error but the deletion does not work.

I have a feeling this has something to do with transaction control.

[koxudaxi]

I found the problem. please wait for a moment...

I explain the problem

I noticed that on version 0.4.0 with our PGDialect change, I still get this error, but the deletion does work. On the newest version, 0.4.2 or even 0.4.1, I also get this error but the deletion does not work.

The behavior is correct. alembic think the delete operation is failed. because rowcount(https://www.python.org/dev/peps/pep-0249/#rowcount) is 0 on response. We have to add the interface. I'm implementing now...

[koxudaxi]

@Rubyj I have released version 0.4.3 which is fixed rowcount when calling delete operation. Would you please check the version?

[Rubyj]

@koxudaxi Yes! It works! Perfect!

Thank you again!

[koxudaxi]

Thank you for your feedback. We need a lot of feedback. If someone post about problems here then, we make the library great things. :wink:

[Rubyj]

@koxudaxi From inside lambda function, I seem to get the following error when I call engine = create_engine('postgresql+pydataapi://', connect_args=aws_args):

{
  "errorType": "NoSuchModuleError",
  "errorMessage": "Can't load plugin: sqlalchemy.dialects:postgresql.pydataapi",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.7/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
    "  File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
    "  File \"\u003cfrozen importlib._bootstrap\u003e\", line 696, in _load\n",
    "  File \"\u003cfrozen importlib._bootstrap\u003e\", line 677, in _load_unlocked\n",
    "  File \"\u003cfrozen importlib._bootstrap_external\u003e\", line 728, in exec_module\n",
    "  File \"\u003cfrozen importlib._bootstrap\u003e\", line 219, in _call_with_frames_removed\n",
    "  File \"/var/task/app.py\", line 11, in \u003cmodule\u003e\n    from chalicelib.session import session_scope\n",
    "  File \"/var/task/chalicelib/session.py\", line 11, in \u003cmodule\u003e\n    engine = create_engine('postgresql+pydataapi://', connect_args=aws_args)\n",
    "  File \"/var/task/sqlalchemy/engine/__init__.py\", line 450, in create_engine\n    return strategy.create(*args, **kwargs)\n",
    "  File \"/var/task/sqlalchemy/engine/strategies.py\", line 61, in create\n    entrypoint = u._get_entrypoint()\n",
    "  File \"/var/task/sqlalchemy/engine/url.py\", line 172, in _get_entrypoint\n    cls = registry.load(name)\n",
    "  File \"/var/task/sqlalchemy/util/langhelpers.py\", line 240, in load\n    \"Can't load plugin: %s:%s\" % (self.group, name)\n"
  ]
}

Any thoughts? I am using aws-sam-cli

[koxudaxi]

@Rubyj
Sorry, I have not used aws-sam-cli I have tested the library with chalice for testing quickly. It works fine.

    from sqlalchemy.engine import create_engine
    engine = create_engine(
        'mysql+pydataapi://',
        connect_args=connect_args)
    result: ResultProxy = engine.execute("select * from pets")
    return result

I guess the problem isｓ installing python's package(eg: pydataapi). Or, Path may be invalid to dialects.

Would you please write how to install packages?

[Rubyj]

@koxudaxi good to know it works with chalice. We deploy with chalice deploy. For local development we use aws-sam-cli because chalice can be limited in it's local testing.

For sam all you need is a template.yaml file in the root project directory. This file should lay out the lambda function(s) in your project code. When you run a sam build on the command line, in your project, it builds based off this template file and it will retrieve any dependencies in the requirements.txt. AWS Toolkit brings this functionality to PyCharm so you can make lambda/sam run configurations in PyCharm projects. It is late now, but I will post some examples in the morning. I have a feeling you are correct that it is not installing or packaging the dependency properly.

[Rubyj]

Here is an example of a HelloWorld sam project:

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  AWS

  Sample SAM Template for AWS

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.7
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

hello_world/app.py

import json

# import requests

def lambda_handler(event, context):

    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "hello world",
            # "location": ip.text.replace("\n", "")
        }),
    }

hello_world/requirements.txt (this is where you could add dependencies and they would be built by sam)

requests

event.json (test event)

{
  "body": "{\"message\": \"hello world\"}",
  "resource": "/{proxy+}",
  "path": "/path/to/resource",
  "httpMethod": "POST",
  "isBase64Encoded": false,
  "queryStringParameters": {
    "foo": "bar"
  },
  "pathParameters": {
    "proxy": "/path/to/resource"
  },
  "stageVariables": {
    "baz": "qux"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "US",
    "Host": "1234567890.execute-api.us-east-1.amazonaws.com",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Custom User Agent String",
    "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==",
    "X-Forwarded-For": "127.0.0.1, 127.0.0.2",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "requestContext": {
    "accountId": "123456789012",
    "resourceId": "123456",
    "stage": "prod",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "requestTime": "09/Apr/2015:12:34:56 +0000",
    "requestTimeEpoch": 1428582896000,
    "identity": {
      "cognitoIdentityPoolId": null,
      "accountId": null,
      "cognitoIdentityId": null,
      "caller": null,
      "accessKey": null,
      "sourceIp": "127.0.0.1",
      "cognitoAuthenticationType": null,
      "cognitoAuthenticationProvider": null,
      "userArn": null,
      "userAgent": "Custom User Agent String",
      "user": null
    },
    "path": "/prod/path/to/resource",
    "resourcePath": "/{proxy+}",
    "httpMethod": "POST",
    "apiId": "1234567890",
    "protocol": "HTTP/1.1"
  }
}

[Rubyj]

Also, do you think that sqlalchemy session will work or probably not?

I am trying to get session.add(model) to work.

[koxudaxi]

@Rubyj

Also, do you think that sqlalchemy session will work or probably not?

I think It will work.

[Rubyj]

@koxudaxi I am trying now within chalice because sam seems troublesome, but seem to be having issues. This is what I am doing.

from sqlalchemy.orm import sessionmaker
from sqlalchemy import create_engine
from contextlib import contextmanager

# configure Session class with desired options
Session = sessionmaker()
# later, we create the engine
aws_args = {'resource_arn': 'arn....',
            'secret_arn': 'arn....,
            'database': 'postgres'}
engine = create_engine('postgresql+pydataapi://', connect_args=aws_args)
# associate it with our custom Session class
Session.configure(bind=engine)

@contextmanager
def session_scope():
    session = Session()
    try:
        yield session
        session.commit()
    except:
        session.rollback()
        raise
    finally:
        session.close()

class BulkInsert(object):
    def go(self, session, inserts):
        session.bulk_save_objects(inserts)

app.py - error occurs on these lines.

    with session_scope() as session:
        BulkInsert().go(session, variants)

I investigate the stack trace and I see:

  Traceback (most recent call last):
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/chalice/app.py", line 1082, in _get_view_function_response
    response = view_function(**function_args)
  File "/home/rjacobs/git/vcfparserlambda/src/parser/app.py", line 26, in index
    parse_vcf(n)
  File "/home/rjacobs/git/vcfparserlambda/src/parser/app.py", line 39, in parse_vcf
    BulkInsert().go(session, variants)
  File "/home/rjacobs/git/vcfparserlambda/src/parser/chalicelib/queries.py", line 3, in go
    session.bulk_save_objects(inserts)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2700, in bulk_save_objects
    False,
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2888, in _bulk_save_mappings
    transaction.rollback(_capture_exception=True)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2882, in _bulk_save_mappings
    render_nulls,
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 102, in _bulk_insert
    bookkeeping=return_defaults,
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 1084, in _emit_insert_statements
    c = cached_connections[connection].execute(statement, multiparams)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
    distilled_params,
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1468, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1224, in _execute_context
    cursor, statement, parameters, context
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 549, in do_executemany
    cursor.executemany(statement, parameters)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/pydataapi/dbapi.py", line 140, in executemany
    results = self._data_api.batch_execute(operation, seq_of_parameters)
  File "/home/rjacobs/git/vcfparserlambda/venv/lib/python3.7/site-packages/pydataapi/pydataapi.py", line 395, in batch_execute
    sql=query,
  File "pydantic/main.py", line 249, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 66362 validation errors for Options
parameterSets -> 0
  value is not a valid list (type=type_error.list)
parameterSets -> 1
  value is not a valid list (type=type_error.list)
parameterSets -> 2
  value is not a valid list (type=type_error.list)
parameterSets -> 3
  value is not a valid list (type=type_error.list)
parameterSets -> 4
  value is not a valid list (type=type_error.list)
parameterSets -> 5
  value is not a valid list (type=type_error.list)
parameterSets -> 6
  value is not a valid list (type=type_error.list)
parameterSets -> 7
  value is not a valid list (type=type_error.list)
parameterSets -> 8
  value is not a valid list (type=type_error.list)
parameterSets -> 9
  value is not a valid list (type=type_error.list)
parameterSets -> 10
  value is not a valid list (type=type_error.list)
parameterSets -> 11
  value is not a valid list (type=type_error.list)
parameterSets -> 12
  value is not a valid list (type=type_error.list)
parameterSets -> 13
  value is not a valid list (type=type_error.list)
parameterSets -> 14
  value is not a valid list (type=type_error.list)
parameterSets -> 15
  value is not a valid list (type=type_error.list)
parameterSets -> 16
  value is not a valid list (type=type_error.list)
parameterSets -> 17
  value is not a valid list (type=type_error.list)
parameterSets -> 18
  value is not a valid list (type=type_error.list)
parameterSets -> 19
  value is not a valid list (type=type_error.list)
parameterSets -> 20
  value is not a valid list (type=type_error.list)
parameterSets -> 21
  value is not a valid list (type=type_error.list)
parameterSets -> 22
  value is not a valid list (type=type_error.list)
parameterSets -> 23
  value is not a valid list (type=type_error.list)
parameterSets -> 24
  value is not a valid list (type=type_error.list)
parameterSets -> 25
  value is not a valid list (type=type_error.list)
parameterSets -> 26
  value is not a valid list (type=type_error.list)
parameterSets -> 27
  value is not a valid list (type=type_error.list)
parameterSets -> 28
  value is not a valid list (type=type_error.list)
parameterSets -> 29
  value is not a valid list (type=type_error.list)
parameterSets -> 30
  value is not a valid list (type=type_error.list)
parameterSets -> 31
  value is not a valid list (type=type_error.list)
parameterSets -> 32
  value is not a valid list (type=type_error.list)
....

FYI these models do have foreign key from one to other. Is this an issue with my models? This is what they look like:

class Variant(Base):
    __tablename__ = 'variant'

    id = Column(Integer, primary_key=True)
    chrom = Column(String(5))
    pos = Column(String(20))
    rs_id = Column(String(50))
    ref = Column(String(50))
    alt = Column(String(50))
    qual = Column(Float)

class VariantAnnotation(Base):
    __tablename__ = 'variant_annotation'

    id = Column(Integer, primary_key=True)
    variant_id = Column(Integer, ForeignKey('variant.id'))
    variant = relationship('Variant', backref='annotations')
    allele = Column(String(50))
    annotation = Column(String(50))
    annotation_impact = Column(String(50))
    gene_name = Column(String(50))
    gene_id = Column(String(50))
    hgvs_c = Column(String(100))
    hgvs_p = Column(String(100))
    reference = Column(String(500))

I build two lists of models to insert like so:

    variants = []
    annotations = []
    for index, row in df.iterrows():
        variant = Variant(chrom=row.CHROM, pos=row.POS, rs_id=row.ID, ref=row.REF, alt=row.ALT_1, qual=row.QUAL)
        annotation = VariantAnnotation(variant=variant, allele=row.ANN_Allele, annotation=row.ANN_Annotation,
                                       annotation_impact=row.ANN_Annotation_Impact, gene_name=row.ANN_Gene_Name,
                                       gene_id=row.ANN_Gene_ID, hgvs_c=row.ANN_HGVS_c, hgvs_p=row.ANN_HGVS_p,
                                       reference=row.ANN_reference)
        variants.append(variant)
        annotations.append(annotation)

These are the lists that are passed to BulkInsert().go(). First the variants list, then the annotation list which has the reference to the other model.

[koxudaxi]

Thank you for your reporting It seems pydantic validation error. I will fix parameterSets which is the request of batch execution.

Btw, I'm creating a sam project in my aws account.

[koxudaxi]

pydantic.error_wrappers.ValidationError: 66362 validation errors for Options parameterSets -> 0

I understand why the errors happen. DataAPI requires Key and value to bulk-insert. But, SQL Alchemy passes only value. I'm fixing it now..

[Rubyj]

pydantic.error_wrappers.ValidationError: 66362 validation errors for Options parameterSets -> 0

I understand why the errors happen. DataAPI requires Key and value to bulk-insert. But, SQL Alchemy passes only value. I'm fixing it now..

Excellent! Thank you very much for fixing things so fast!

If you would like me to look at your sam implementation or to help you with that I am more than happy to.

[Rubyj]

@koxudaxi I deployed function to AWS using chalice deploy. When runs on AWS I get:

{
  "errorMessage": "Can't load plugin: sqlalchemy.dialects:postgresql.pydataapi",
  "errorType": "NoSuchModuleError",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.7/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
    "  File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
    "  File \"<frozen importlib._bootstrap>\", line 696, in _load\n",
    "  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
    "  File \"/var/task/app.py\", line 10, in <module>\n    from chalicelib.session import session_scope\n",
    "  File \"/var/task/chalicelib/session.py\", line 11, in <module>\n    engine = create_engine('postgresql+pydataapi://', connect_args=aws_args)\n",
    "  File \"/var/task/sqlalchemy/engine/__init__.py\", line 450, in create_engine\n    return strategy.create(*args, **kwargs)\n",
    "  File \"/var/task/sqlalchemy/engine/strategies.py\", line 61, in create\n    entrypoint = u._get_entrypoint()\n",
    "  File \"/var/task/sqlalchemy/engine/url.py\", line 172, in _get_entrypoint\n    cls = registry.load(name)\n",
    "  File \"/var/task/sqlalchemy/util/langhelpers.py\", line 240, in load\n    \"Can't load plugin: %s:%s\" % (self.group, name)\n"
  ]
}

:cry:

Chalice was unable to install sqlalchemy so I had to manually include it and pydataapi in vendor/ directory. When installed like this it seems that pydataapi does not detect install of sqlalchemy

[koxudaxi]

@Rubyj

Chalice was unable to install sqlalchemy so I had to manually include it and pydataapi in vendor/ directory. When installed like this it seems that pydataapi does not detect install of sqlalchemy

you may need a file entry_points.txt in pydataapi-xxx.dist-info

Could you tell me what your OS and OS version? I can deploy with a chalice in macOS.

[Rubyj]

@koxudaxi.

I am using chalice on Manjaro Linux. What does that file do? I can try to add that in the morning.

When you do a chalice deploy it is able to install sqlalchemy in requirements.txt? It is unable to install for me due to C extensions of sqlalchemy I believe.

[koxudaxi]

When you do a chalice deploy it is able to install sqlalchemy in requirements.txt? It is unable to install for me due to C extensions of sqlalchemy I believe.

It may correct. Could you use docker to build a package in the chalice? or sam?

$ cat requirements.txt
pydataapi

After do chalice deploy

 zipinfo -l .chalice/deployments/xxxxxxxx-python3.7.zip |grep -E 'pydataapi|sqlalchemy'
-rw-r--r--  2.0 unx        1 b-        3 defN 19-Oct-17 14:01 pydataapi-0.4.4.dist-info/zip-safe
-rw-r--r--  2.0 unx     1729 b-      920 defN 19-Oct-17 14:01 pydataapi-0.4.4.dist-info/RECORD
-rw-r--r--  2.0 unx     1068 b-      629 defN 19-Oct-17 14:01 pydataapi-0.4.4.dist-info/LICENSE
-rw-r--r--  2.0 unx       92 b-       92 defN 19-Oct-17 14:01 pydataapi-0.4.4.dist-info/WHEEL
-rw-r--r--  2.0 unx      145 b-       76 defN 19-Oct-17 14:01 pydataapi-0.4.4.dist-info/entry_points.txt
-rw-r--r--  2.0 unx       16 b-       18 defN 19-Oct-17 14:01 pydataapi-0.4.4.dist-info/top_level.txt
-rw-r--r--  2.0 unx     8391 b-     2592 defN 19-Oct-17 14:01 pydataapi-0.4.4.dist-info/METADATA
-rw-r--r--  2.0 unx    12636 b-     1364 defN 19-Oct-17 14:01 tests/pydataapi/test_dialect.py
-rw-r--r--  2.0 unx        0 b-        2 defN 19-Oct-17 14:01 tests/pydataapi/__init__.py
-rw-r--r--  2.0 unx    20461 b-     3153 defN 19-Oct-17 14:01 tests/pydataapi/test_pydataapi.py
-rw-r--r--  2.0 unx    10625 b-     1342 defN 19-Oct-17 14:01 tests/pydataapi/test_dbaapi.py
-rw-r--r--  2.0 unx    12721 b-     3233 defN 19-Oct-17 14:01 sqlalchemy/interfaces.py
-rw-r--r--  2.0 unx     6693 b-     2187 defN 19-Oct-17 14:01 sqlalchemy/log.py
-rw-r--r--  2.0 unx    51669 b-    11668 defN 19-Oct-17 14:01 sqlalchemy/events.py
-rw-r--r--  2.0 unx     4621 b-     1096 defN 19-Oct-17 14:01 sqlalchemy/__init__.py
-rw-r--r--  2.0 unx     3377 b-      770 defN 19-Oct-17 14:01 sqlalchemy/types.py
-rw-r--r--  2.0 unx     2990 b-     1283 defN 19-Oct-17 14:01 sqlalchemy/inspection.py
-rw-r--r--  2.0 unx    16088 b-     4721 defN 19-Oct-17 14:01 sqlalchemy/exc.py
-rw-r--r--  2.0 unx     5648 b-     1449 defN 19-Oct-17 14:01 sqlalchemy/processors.py
-rw-r--r--  2.0 unx     2377 b-      583 defN 19-Oct-17 14:01 sqlalchemy/schema.py
-rw-r--r--  2.0 unx     5352 b-     2038 defN 19-Oct-17 14:01 sqlalchemy/connectors/mxodbc.py
-rw-r--r--  2.0 unx     1878 b-      833 defN 19-Oct-17 14:01 sqlalchemy/connectors/zxJDBC.py
-rw-r--r--  2.0 unx      278 b-      211 defN 19-Oct-17 14:01 sqlalchemy/connectors/__init__.py
-rw-r--r--  2.0 unx     5586 b-     1943 defN 19-Oct-17 14:01 sqlalchemy/connectors/pyodbc.py
-rw-r--r--  2.0 unx      819 b-      382 defN 19-Oct-17 14:01 sqlalchemy/databases/__init__.py
-rw-r--r--  2.0 unx     6827 b-     2005 defN 19-Oct-17 14:01 sqlalchemy/util/queue.py
-rw-r--r--  2.0 unx     2767 b-     1090 defN 19-Oct-17 14:01 sqlalchemy/util/topological.py
-rw-r--r--  2.0 unx    11794 b-     3782 defN 19-Oct-17 14:01 sqlalchemy/util/compat.py
-rw-r--r--  2.0 unx     6580 b-     1399 defN 19-Oct-17 14:01 sqlalchemy/util/__init__.py
-rw-r--r--  2.0 unx     7128 b-     1793 defN 19-Oct-17 14:01 sqlalchemy/util/deprecations.py
-rw-r--r--  2.0 unx    29153 b-     7067 defN 19-Oct-17 14:01 sqlalchemy/util/_collections.py
-rw-r--r--  2.0 unx    49149 b-    14311 defN 19-Oct-17 14:01 sqlalchemy/util/langhelpers.py
-rw-r--r--  2.0 unx    14351 b-     3687 defN 19-Oct-17 14:01 sqlalchemy/ext/instrumentation.py
-rw-r--r--  2.0 unx    16850 b-     5565 defN 19-Oct-17 14:01 sqlalchemy/ext/compiler.py
-rw-r--r--  2.0 unx    13888 b-     4528 defN 19-Oct-17 14:01 sqlalchemy/ext/orderinglist.py
-rw-r--r--  2.0 unx    40233 b-    10170 defN 19-Oct-17 14:01 sqlalchemy/ext/hybrid.py
-rw-r--r--  2.0 unx      322 b-      239 defN 19-Oct-17 14:01 sqlalchemy/ext/__init__.py
-rw-r--r--  2.0 unx    11169 b-     3493 defN 19-Oct-17 14:01 sqlalchemy/ext/indexable.py
-rw-r--r--  2.0 unx     5784 b-     1999 defN 19-Oct-17 14:01 sqlalchemy/ext/serializer.py
-rw-r--r--  2.0 unx    21469 b-     5887 defN 19-Oct-17 14:01 sqlalchemy/ext/baked.py
-rw-r--r--  2.0 unx    41710 b-    10643 defN 19-Oct-17 14:01 sqlalchemy/ext/automap.py
-rw-r--r--  2.0 unx     9109 b-     2653 defN 19-Oct-17 14:01 sqlalchemy/ext/horizontal_shard.py
-rw-r--r--  2.0 unx    31808 b-     7951 defN 19-Oct-17 14:01 sqlalchemy/ext/mutable.py
-rw-r--r--  2.0 unx    49463 b-    10398 defN 19-Oct-17 14:01 sqlalchemy/ext/associationproxy.py
-rw-r--r--  2.0 unx    11051 b-     3166 defN 19-Oct-17 14:01 sqlalchemy/ext/declarative/clsregistry.py
-rw-r--r--  2.0 unx      902 b-      370 defN 19-Oct-17 14:01 sqlalchemy/ext/declarative/__init__.py
-rw-r--r--  2.0 unx    27683 b-     7801 defN 19-Oct-17 14:01 sqlalchemy/ext/declarative/api.py
-rw-r--r--  2.0 unx    32093 b-     7633 defN 19-Oct-17 14:01 sqlalchemy/ext/declarative/base.py
-rw-r--r--  2.0 unx     2693 b-      700 defN 19-Oct-17 14:01 sqlalchemy/testing/pickleable.py
-rw-r--r--  2.0 unx    12765 b-     3075 defN 19-Oct-17 14:01 sqlalchemy/testing/exclusions.py
-rw-r--r--  2.0 unx     2680 b-      926 defN 19-Oct-17 14:01 sqlalchemy/testing/config.py
-rw-r--r--  2.0 unx    10437 b-     3057 defN 19-Oct-17 14:01 sqlalchemy/testing/engines.py
-rw-r--r--  2.0 unx     7738 b-     2621 defN 19-Oct-17 14:01 sqlalchemy/testing/util.py
-rw-r--r--  2.0 unx     1298 b-      536 defN 19-Oct-17 14:01 sqlalchemy/testing/warnings.py
-rw-r--r--  2.0 unx     2362 b-      630 defN 19-Oct-17 14:01 sqlalchemy/testing/__init__.py
-rw-r--r--  2.0 unx    18017 b-     5023 defN 19-Oct-17 14:01 sqlalchemy/testing/assertions.py
-rw-r--r--  2.0 unx     5875 b-     1323 defN 19-Oct-17 14:01 sqlalchemy/testing/replay_fixture.py
-rw-r--r--  2.0 unx    13398 b-     3018 defN 19-Oct-17 14:01 sqlalchemy/testing/assertsql.py
-rw-r--r--  2.0 unx    13263 b-     3228 defN 19-Oct-17 14:01 sqlalchemy/testing/provision.py
-rw-r--r--  2.0 unx    27126 b-     6172 defN 19-Oct-17 14:01 sqlalchemy/testing/requirements.py
-rw-r--r--  2.0 unx     3203 b-     1050 defN 19-Oct-17 14:01 sqlalchemy/testing/entities.py
-rw-r--r--  2.0 unx     8513 b-     2719 defN 19-Oct-17 14:01 sqlalchemy/testing/profiling.py
-rw-r--r--  2.0 unx    10814 b-     2665 defN 19-Oct-17 14:01 sqlalchemy/testing/fixtures.py
-rw-r--r--  2.0 unx     3712 b-     1414 defN 19-Oct-17 14:01 sqlalchemy/testing/schema.py
-rw-r--r--  2.0 unx      893 b-      393 defN 19-Oct-17 14:01 sqlalchemy/testing/mock.py
-rw-r--r--  2.0 unx     1468 b-      699 defN 19-Oct-17 14:01 sqlalchemy/testing/plugin/bootstrap.py
-rw-r--r--  2.0 unx    19692 b-     5107 defN 19-Oct-17 14:01 sqlalchemy/testing/plugin/plugin_base.py
-rw-r--r--  2.0 unx        0 b-        2 defN 19-Oct-17 14:01 sqlalchemy/testing/plugin/__init__.py
-rw-r--r--  2.0 unx     7166 b-     2254 defN 19-Oct-17 14:01 sqlalchemy/testing/plugin/pytestplugin.py
-rw-r--r--  2.0 unx     4661 b-      986 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_sequence.py
-rw-r--r--  2.0 unx    11504 b-     2342 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_results.py
-rw-r--r--  2.0 unx     9586 b-     1552 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_insert.py
-rw-r--r--  2.0 unx    19984 b-     2656 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_select.py
-rw-r--r--  2.0 unx     2870 b-      641 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_ddl.py
-rw-r--r--  2.0 unx     4572 b-     1395 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_dialect.py
-rw-r--r--  2.0 unx      358 b-       98 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/__init__.py
-rw-r--r--  2.0 unx     1478 b-      486 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_update_delete.py
-rw-r--r--  2.0 unx    38495 b-     7476 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_reflection.py
-rw-r--r--  2.0 unx     6788 b-     1055 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_cte.py
-rw-r--r--  2.0 unx    30448 b-     5464 defN 19-Oct-17 14:01 sqlalchemy/testing/suite/test_types.py
-rw-r--r--  2.0 unx     1349 b-      615 defN 19-Oct-17 14:01 sqlalchemy/dialects/__init__.py
-rw-r--r--  2.0 unx      902 b-      435 defN 19-Oct-17 14:01 sqlalchemy/dialects/sybase/mxodbc.py
-rw-r--r--  2.0 unx     1363 b-      464 defN 19-Oct-17 14:01 sqlalchemy/dialects/sybase/__init__.py
-rw-r--r--  2.0 unx     3313 b-     1268 defN 19-Oct-17 14:01 sqlalchemy/dialects/sybase/pysybase.py
-rw-r--r--  2.0 unx     2120 b-      881 defN 19-Oct-17 14:01 sqlalchemy/dialects/sybase/pyodbc.py
-rw-r--r--  2.0 unx    31953 b-     7471 defN 19-Oct-17 14:01 sqlalchemy/dialects/sybase/base.py
-rw-r--r--  2.0 unx     1415 b-      641 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/zxjdbc.py
-rw-r--r--  2.0 unx     1657 b-      775 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/psycopg2cffi.py
-rw-r--r--  2.0 unx     2461 b-      767 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/__init__.py
-rw-r--r--  2.0 unx     8129 b-     2029 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/pygresql.py
-rw-r--r--  2.0 unx     4750 b-     1246 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/ranges.py
-rw-r--r--  2.0 unx     2915 b-     1188 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/pypostgresql.py
-rw-r--r--  2.0 unx    35528 b-    10588 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/psycopg2.py
-rw-r--r--  2.0 unx     9719 b-     2946 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/pg8000.py
-rw-r--r--  2.0 unx    11089 b-     3392 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/array.py
-rw-r--r--  2.0 unx     7579 b-     2111 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/dml.py
-rw-r--r--  2.0 unx     9938 b-     2878 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/json.py
-rw-r--r--  2.0 unx   119131 b-    28992 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/base.py
-rw-r--r--  2.0 unx     6823 b-     2339 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/ext.py
-rw-r--r--  2.0 unx    12510 b-     3672 defN 19-Oct-17 14:01 sqlalchemy/dialects/postgresql/hstore.py
-rw-r--r--  2.0 unx     8207 b-     2490 defN 19-Oct-17 14:01 sqlalchemy/dialects/oracle/zxjdbc.py
-rw-r--r--  2.0 unx     1257 b-      453 defN 19-Oct-17 14:01 sqlalchemy/dialects/oracle/__init__.py
-rw-r--r--  2.0 unx    40689 b-    10622 defN 19-Oct-17 14:01 sqlalchemy/dialects/oracle/cx_oracle.py
-rw-r--r--  2.0 unx    67114 b-    16454 defN 19-Oct-17 14:01 sqlalchemy/dialects/oracle/base.py
-rw-r--r--  2.0 unx     4689 b-     1870 defN 19-Oct-17 14:01 sqlalchemy/dialects/sqlite/pysqlcipher.py
-rw-r--r--  2.0 unx     1042 b-      425 defN 19-Oct-17 14:01 sqlalchemy/dialects/sqlite/__init__.py
-rw-r--r--  2.0 unx     2292 b-      847 defN 19-Oct-17 14:01 sqlalchemy/dialects/sqlite/json.py
-rw-r--r--  2.0 unx    70302 b-    17464 defN 19-Oct-17 14:01 sqlalchemy/dialects/sqlite/base.py
-rw-r--r--  2.0 unx    14878 b-     5201 defN 19-Oct-17 14:01 sqlalchemy/dialects/sqlite/pysqlite.py
-rw-r--r--  2.0 unx    11307 b-     3276 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/enumerated.py
-rw-r--r--  2.0 unx     8086 b-     2309 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/oursql.py
-rw-r--r--  2.0 unx     7889 b-     2380 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/mysqlconnector.py
-rw-r--r--  2.0 unx    17837 b-     5183 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/reflection.py
-rw-r--r--  2.0 unx     3970 b-     1745 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/zxjdbc.py
-rw-r--r--  2.0 unx     2056 b-      614 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/__init__.py
-rw-r--r--  2.0 unx    24601 b-     3641 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/types.py
-rw-r--r--  2.0 unx     8383 b-     2805 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/mysqldb.py
-rw-r--r--  2.0 unx     2338 b-     1023 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/pymysql.py
-rw-r--r--  2.0 unx     3368 b-     1390 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/gaerdbms.py
-rw-r--r--  2.0 unx     4559 b-     1703 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/dml.py
-rw-r--r--  2.0 unx     2050 b-      768 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/json.py
-rw-r--r--  2.0 unx     2691 b-     1221 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/pyodbc.py
-rw-r--r--  2.0 unx    97424 b-    25028 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/base.py
-rw-r--r--  2.0 unx     2245 b-      967 defN 19-Oct-17 14:01 sqlalchemy/dialects/mysql/cymysql.py
-rw-r--r--  2.0 unx     4616 b-     1792 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/mxodbc.py
-rw-r--r--  2.0 unx     2311 b-     1006 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/zxjdbc.py
-rw-r--r--  2.0 unx     1812 b-      568 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/__init__.py
-rw-r--r--  2.0 unx     3913 b-     1623 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/pymssql.py
-rw-r--r--  2.0 unx     2719 b-     1092 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/adodbapi.py
-rw-r--r--  2.0 unx     5343 b-     1334 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/information_schema.py
-rw-r--r--  2.0 unx    12284 b-     4183 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/pyodbc.py
-rw-r--r--  2.0 unx    86293 b-    21410 defN 19-Oct-17 14:01 sqlalchemy/dialects/mssql/base.py
-rw-r--r--  2.0 unx     1152 b-      387 defN 19-Oct-17 14:01 sqlalchemy/dialects/firebird/__init__.py
-rw-r--r--  2.0 unx     4071 b-     1846 defN 19-Oct-17 14:01 sqlalchemy/dialects/firebird/fdb.py
-rw-r--r--  2.0 unx     6372 b-     2286 defN 19-Oct-17 14:01 sqlalchemy/dialects/firebird/kinterbasdb.py
-rw-r--r--  2.0 unx    30566 b-     8036 defN 19-Oct-17 14:01 sqlalchemy/dialects/firebird/base.py
-rw-r--r--  2.0 unx     5536 b-     1486 defN 19-Oct-17 14:01 sqlalchemy/orm/sync.py
-rw-r--r--  2.0 unx    25766 b-     7411 defN 19-Oct-17 14:01 sqlalchemy/orm/interfaces.py
-rw-r--r--  2.0 unx    18131 b-     5075 defN 19-Oct-17 14:01 sqlalchemy/orm/instrumentation.py
-rw-r--r--  2.0 unx    32236 b-     8314 defN 19-Oct-17 14:01 sqlalchemy/orm/loading.py
-rw-r--r--  2.0 unx    65652 b-    12247 defN 19-Oct-17 14:01 sqlalchemy/orm/persistence.py
-rw-r--r--  2.0 unx     9262 b-     2759 defN 19-Oct-17 14:01 sqlalchemy/orm/path_registry.py
-rw-r--r--  2.0 unx   172794 b-    41404 defN 19-Oct-17 14:01 sqlalchemy/orm/query.py
-rw-r--r--  2.0 unx    67137 b-    14471 defN 19-Oct-17 14:01 sqlalchemy/orm/attributes.py
-rw-r--r--  2.0 unx    85055 b-    18225 defN 19-Oct-17 14:01 sqlalchemy/orm/strategies.py
-rw-r--r--  2.0 unx    55505 b-    12531 defN 19-Oct-17 14:01 sqlalchemy/orm/strategy_options.py
-rw-r--r--  2.0 unx    11043 b-     3605 defN 19-Oct-17 14:01 sqlalchemy/orm/properties.py
-rw-r--r--  2.0 unx    44859 b-    12047 defN 19-Oct-17 14:01 sqlalchemy/orm/util.py
-rw-r--r--  2.0 unx    94409 b-    17826 defN 19-Oct-17 14:01 sqlalchemy/orm/events.py
-rw-r--r--  2.0 unx   128344 b-    29279 defN 19-Oct-17 14:01 sqlalchemy/orm/mapper.py
-rw-r--r--  2.0 unx     6393 b-     2137 defN 19-Oct-17 14:01 sqlalchemy/orm/scoping.py
-rw-r--r--  2.0 unx   128754 b-    30903 defN 19-Oct-17 14:01 sqlalchemy/orm/session.py
-rw-r--r--  2.0 unx     9542 b-     3092 defN 19-Oct-17 14:01 sqlalchemy/orm/__init__.py
-rw-r--r--  2.0 unx    14666 b-     3484 defN 19-Oct-17 14:01 sqlalchemy/orm/dynamic.py
-rw-r--r--  2.0 unx     6610 b-     2172 defN 19-Oct-17 14:01 sqlalchemy/orm/exc.py
-rw-r--r--  2.0 unx    52614 b-    12524 defN 19-Oct-17 14:01 sqlalchemy/orm/collections.py
-rw-r--r--  2.0 unx    46556 b-     7441 defN 19-Oct-17 14:01 sqlalchemy/orm/dependency.py
-rw-r--r--  2.0 unx    20751 b-     4348 defN 19-Oct-17 14:01 sqlalchemy/orm/deprecated_interfaces.py
-rw-r--r--  2.0 unx    28201 b-     7378 defN 19-Oct-17 14:01 sqlalchemy/orm/descriptor_props.py
-rw-r--r--  2.0 unx    10299 b-     2041 defN 19-Oct-17 14:01 sqlalchemy/orm/identity.py
-rw-r--r--  2.0 unx     5441 b-     1404 defN 19-Oct-17 14:01 sqlalchemy/orm/evaluator.py
-rw-r--r--  2.0 unx    24735 b-     5665 defN 19-Oct-17 14:01 sqlalchemy/orm/unitofwork.py
-rw-r--r--  2.0 unx    14864 b-     4346 defN 19-Oct-17 14:01 sqlalchemy/orm/base.py
-rw-r--r--  2.0 unx   124442 b-    27251 defN 19-Oct-17 14:01 sqlalchemy/orm/relationships.py
-rw-r--r--  2.0 unx    31044 b-     7763 defN 19-Oct-17 14:01 sqlalchemy/orm/state.py
-rw-r--r--  2.0 unx    42397 b-    10159 defN 19-Oct-17 14:01 sqlalchemy/engine/interfaces.py
-rw-r--r--  2.0 unx     4746 b-     1311 defN 19-Oct-17 14:01 sqlalchemy/engine/threadlocal.py
-rw-r--r--  2.0 unx     9750 b-     2668 defN 19-Oct-17 14:01 sqlalchemy/engine/strategies.py
-rw-r--r--  2.0 unx     2421 b-      905 defN 19-Oct-17 14:01 sqlalchemy/engine/util.py
-rw-r--r--  2.0 unx    33343 b-     6965 defN 19-Oct-17 14:01 sqlalchemy/engine/reflection.py
-rw-r--r--  2.0 unx    22355 b-     7381 defN 19-Oct-17 14:01 sqlalchemy/engine/__init__.py
-rw-r--r--  2.0 unx     9411 b-     2951 defN 19-Oct-17 14:01 sqlalchemy/engine/url.py
-rw-r--r--  2.0 unx    52615 b-    11873 defN 19-Oct-17 14:01 sqlalchemy/engine/result.py
-rw-r--r--  2.0 unx    51401 b-    11571 defN 19-Oct-17 14:01 sqlalchemy/engine/default.py
-rw-r--r--  2.0 unx    84126 b-    17996 defN 19-Oct-17 14:01 sqlalchemy/engine/base.py
-rw-r--r--  2.0 unx     4320 b-     1584 defN 19-Oct-17 14:01 sqlalchemy/pool/dbapi_proxy.py
-rw-r--r--  2.0 unx     1483 b-      656 defN 19-Oct-17 14:01 sqlalchemy/pool/__init__.py
-rw-r--r--  2.0 unx    14594 b-     3797 defN 19-Oct-17 14:01 sqlalchemy/pool/impl.py
-rw-r--r--  2.0 unx    34609 b-     8827 defN 19-Oct-17 14:01 sqlalchemy/pool/base.py
-rw-r--r--  2.0 unx     5904 b-     1655 defN 19-Oct-17 14:01 sqlalchemy/event/legacy.py
-rw-r--r--  2.0 unx     8066 b-     1965 defN 19-Oct-17 14:01 sqlalchemy/event/registry.py
-rw-r--r--  2.0 unx      596 b-      301 defN 19-Oct-17 14:01 sqlalchemy/event/__init__.py
-rw-r--r--  2.0 unx     7085 b-     1917 defN 19-Oct-17 14:01 sqlalchemy/event/api.py
-rw-r--r--  2.0 unx    12994 b-     3185 defN 19-Oct-17 14:01 sqlalchemy/event/attr.py
-rw-r--r--  2.0 unx     9760 b-     2963 defN 19-Oct-17 14:01 sqlalchemy/event/base.py
-rw-r--r--  2.0 unx    33760 b-     8669 defN 19-Oct-17 14:01 sqlalchemy/sql/functions.py
-rw-r--r--  2.0 unx     6651 b-     2239 defN 19-Oct-17 14:01 sqlalchemy/sql/annotation.py
-rw-r--r--  2.0 unx    39549 b-     9265 defN 19-Oct-17 14:01 sqlalchemy/sql/ddl.py
-rw-r--r--  2.0 unx   124358 b-    25545 defN 19-Oct-17 14:01 sqlalchemy/sql/compiler.py
-rw-r--r--  2.0 unx     5885 b-     1761 defN 19-Oct-17 14:01 sqlalchemy/sql/naming.py
-rw-r--r--  2.0 unx    29275 b-     7825 defN 19-Oct-17 14:01 sqlalchemy/sql/util.py
-rw-r--r--  2.0 unx    10290 b-     2871 defN 19-Oct-17 14:01 sqlalchemy/sql/visitors.py
-rw-r--r--  2.0 unx    12234 b-     2847 defN 19-Oct-17 14:01 sqlalchemy/sql/default_comparator.py
-rw-r--r--  2.0 unx   153952 b-    36327 defN 19-Oct-17 14:01 sqlalchemy/sql/elements.py
-rw-r--r--  2.0 unx     3789 b-      825 defN 19-Oct-17 14:01 sqlalchemy/sql/__init__.py
-rw-r--r--  2.0 unx    95377 b-    22962 defN 19-Oct-17 14:01 sqlalchemy/sql/sqltypes.py
-rw-r--r--  2.0 unx    42221 b-     8247 defN 19-Oct-17 14:01 sqlalchemy/sql/operators.py
-rw-r--r--  2.0 unx   132514 b-    29804 defN 19-Oct-17 14:01 sqlalchemy/sql/selectable.py
-rw-r--r--  2.0 unx    34420 b-     8645 defN 19-Oct-17 14:01 sqlalchemy/sql/dml.py
-rw-r--r--  2.0 unx     9037 b-     2253 defN 19-Oct-17 14:01 sqlalchemy/sql/expression.py
-rw-r--r--  2.0 unx    21506 b-     5848 defN 19-Oct-17 14:01 sqlalchemy/sql/base.py
-rw-r--r--  2.0 unx    51485 b-    12957 defN 19-Oct-17 14:01 sqlalchemy/sql/type_api.py
-rw-r--r--  2.0 unx    25840 b-     5676 defN 19-Oct-17 14:01 sqlalchemy/sql/crud.py
-rw-r--r--  2.0 unx   164499 b-    39057 defN 19-Oct-17 14:01 sqlalchemy/sql/schema.py
-rw-r--r--  2.0 unx    13516 b-     2927 defN 19-Oct-17 14:01 pydataapi/pydataapi.py
-rw-r--r--  2.0 unx      116 b-      102 defN 19-Oct-17 14:01 pydataapi/version.py
-rw-r--r--  2.0 unx     5626 b-     1444 defN 19-Oct-17 14:01 pydataapi/dbapi.py
-rw-r--r--  2.0 unx     4861 b-      908 defN 19-Oct-17 14:01 pydataapi/dialect.py
-rw-r--r--  2.0 unx      358 b-      177 defN 19-Oct-17 14:01 pydataapi/__init__.py
-rw-r--r--  2.0 unx      371 b-      167 defN 19-Oct-17 14:01 pydataapi/exceptions.py

[Rubyj]

What would you like me to do with docker?

I have the same requirements file as you. If I try chalice deploy I get:

Could not install dependencies:
sqlalchemy==1.3.10
You will have to build these yourself and vendor them in
the chalice vendor folder

Using chalice==1.12.0

[koxudaxi]

What would you like me to do with docker?

I think chalice can install sqlalchemy in a few containers. But, It may be wrong.

Could you tell me your python version?

hint: https://github.com/aws/chalice/issues/1024

[Rubyj]

I am running python3.7. I think I may need to install with C extensions disabled. I will try this tomorrow and post the results here. Thank you!

I will also see if I can get a debug log of chalice package

[koxudaxi]

Thank you, I will test on x86_64 Linux like Ubuntu tonight.

[koxudaxi]

I found the way to exclude c extensions. $ export DISABLE_SQLALCHEMY_CEXT=1; chalice package pkg https://docs.sqlalchemy.org/en/13/intro.html#installing-the-c-extensions

We should write the information on documents for chalice users.

$ uname -a   
Linux vultr-ubuntu 4.15.0-33-generic #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ python3 --version
Python 3.6.7

$ cat requirements.txt 
sqlalchemy==1.3.10

### install SQLAlchemy with C Extensions
$ chalice package pkg
Creating deployment package.

$ zipinfo pkg/deployment.zip |grep "\.so"
-rw-rw-r--  2.0 unx    34936 b- defN 19-Oct-17 22:34 sqlalchemy/cutils.cpython-36m-x86_64-linux-gnu.so
-rw-rw-r--  2.0 unx    52616 b- defN 19-Oct-17 22:34 sqlalchemy/cprocessors.cpython-36m-x86_64-linux-gnu.so
-rw-rw-r--  2.0 unx    62552 b- defN 19-Oct-17 22:34 sqlalchemy/cresultproxy.cpython-36m-x86_64-linux-gnu.so

### install SQLAlchemy wihtout C Extensions
$ export DISABLE_SQLALCHEMY_CEXT=1;  chalice package pkg
Creating deployment package.

$ zipinfo pkg/deployment.zip |grep "\.so"
# nothing

[Rubyj]

You were able to install sqlalchemy with c extensions with no problems? Very frustrating that I cannot. Maybe I should try python3.6 instead of 3.7

I will try to disable the extensions and install now.

[Rubyj]

@koxudaxi Unfortunately, this still does not work for me :cry:

requirements.txt

pydataapi

$ DISABLE_SQLALCHEMY_CEXT=1 chalice package pkg
Creating deployment package.

Could not install dependencies:
sqlalchemy==1.3.7
You will have to build these yourself and vendor them in
the chalice vendor folder.

Your deployment will continue but may not work correctly
if missing dependencies are not present. For more information:
http://chalice.readthedocs.io/en/latest/topics/packaging.html

What version of chalice do you use?

Using python3.6 I seem to be able to install sqlalchemy I will test using python3.6

[Rubyj]

I was able to deploy with python3.6. However, when I do and run the lambda function I get this error:

Unable to import module 'app': No module named 'sqlalchemy'

Any thoughts?