Testing Zappa apps

[fbidu]

Context

Hello,

Today I got myself almost banging my head against the wall because I've misconfigured app_function in zappa_settings.json. The errors I was getting were cryptic. A simple "NoneType is not callable". Not an import error of some sorts, just that.

I'm a little familiar with that sort of error in Flask applications - my case - so I set out looking for places were I might have been exposing invalid variables and so on. Turns out that instead of pointing to a Flask app I was pointing to the module where my app resides.

Dumb mistake, totally my fault but this is so preventable. Zappa could do something like checking if the string set as app_function is indeed something it can use. It could be done by default before every update or just exposed as some command like zappa test or something.

I took a look around the issues and docs but didn't find anything regarding this kind of behaviour. All in all, I'd be happy to help!

Expected Behavior

Zappa could provide something that checks against errors in the settings or some facilities to allow testing.

Actual Behavior

It does not :(

Possible Fix

Steps to Reproduce

1. 2. 3.

Your Environment

• Zappa version used: 0.47.1
• Operating System and Python version: 3.6.7
• The output of pip freeze:

appdirs==1.4.3 argcomplete==1.9.3 astroid==2.2.0.dev1 atomicwrites==1.3.0 attrs==18.2.0 black==18.9b0 boto3==1.9.102 botocore==1.12.102 cachetools==3.1.0 certifi==2018.11.29 cfn-flip==1.1.0.post1 chardet==3.0.4 Click==7.0 docutils==0.14 durationpy==0.5 Flask==1.0.2 future==0.16.0 google-api-core==1.8.0 google-auth==1.6.3 google-cloud-bigquery==1.9.0 google-cloud-core==0.29.1 google-resumable-media==0.3.2 googleapis-common-protos==1.6.0b9 hjson==3.0.1 idna==2.8 isort==4.3.9 itsdangerous==1.1.0 Jinja2==2.10 jmespath==0.9.3 kappa==0.6.0 lambda-packages==0.20.0 lazy-object-proxy==1.3.1 MarkupSafe==1.1.1 mccabe==0.6.1 more-itertools==6.0.0 placebo==0.9.0 pluggy==0.9.0 protobuf==3.7.0rc3 py==1.8.0 pyasn1==0.4.5 pyasn1-modules==0.2.4 pylint==2.3.0.dev2 pymongo==3.7.2 pytest==4.3.0 pytest-mock==1.10.1 python-dateutil==2.6.1 python-slugify==1.2.4 pytz==2018.9 PyYAML==3.13 requests==2.21.0 rsa==4.0 s3transfer==0.2.0 six==1.12.0 toml==0.10.0 tqdm==4.19.1 troposphere==2.4.5 typed-ast==1.2.0 Unidecode==1.0.23 urllib3==1.24.1 Werkzeug==0.14.1 wrapt==1.11.1 wsgi-request-logger==0.4.6 zappa==0.47.1

• Link to your project (optional):
• Your zappa_settings.py:

[fbidu]

So, I was writing a new zappa project today and decided to create a simple test_zappa.py script. Basically, it checks if there is a settings file, if the functions used by the environments exist and are valid flask.Flask objects, which is my current use case.

I think this code can be improved to handle more generic scenarios and solve this issue

"""
test_zappa performs some checkings in the mechanics of Zappa itself. It checks
if there is a settings file and if the functions used by each environment exists
and are valid.

Currently, this module does not handle paths very well. It must be executed from
the same directory where the possible settings file exists, probably in the root
of your project.
"""
from importlib import import_module
from json import load
from os import getcwd, path

from flask import Flask

current_dir = getcwd()
zappa_settings_file = path.join(current_dir, "zappa_settings.json")

def check_function_is_valid(module_name: str, function_name: str) -> bool:
    """
    Checks if a given `function_name` exists inside a module called `module_name`
    and if that function is an instance of flask.Flask. These are the necessary
    attributes for a given `app_function` key inside a Zappa environment to
    be valid.
    """

    try:
        # Dynamically loading the module
        module = import_module(module_name)
    except ModuleNotFoundError:
        print(
            f"Failure loading module {module_name}! Maybe you're having issues with your PATH"
        )
        return False

    # Does the function exists?
    assert hasattr(
        module, function_name
    ), f"Function {function_name} does not exist at module {module_name}"

    function = getattr(module, function_name)

    # Is it a Flask app?
    assert isinstance(
        function, Flask
    ), f"The function {function_name} was loaded but its type is {type(function_name)} instead of flask.Flask"

def test_zappa_has_settings():
    """
    Tests if there is a `zappa_settings.json` file.
    """

    # Do we even have a Zappa settings?
    assert path.exists(
        zappa_settings_file
    ), "You don't have a zappa_settings file or you're running this script from the wrong folder"

def test_zappa_loading():
    """
    Tests if the environments configured at Zappa Settings have valid `app_function` parameters
    """

    # Loads the settings as a dictionary
    settings = load(open(zappa_settings_file))

    # All the environments defined in the settings file must have an 'app_function' key
    assert all(
        ["app_function" in environment for environment in settings.values()]
    ), "Some of your zappa environments do not have an 'app_function' key"

    app_functions = [environment["app_function"] for environment in settings.values()]

    for function in app_functions:
        module_name = ".".join(function.split(".")[:-1])
        function_name = function.split(".")[-1]
        check_function_is_valid(module_name, function_name)