Chalice gets confused when used in multiple CDK NestedStack

[mdemarqu]

I use 2 Chalice API in a same CDK app, each one is handled in a separate nested stack.

CDKRootStack
 |-ChaliceAPI1
 |-ChaliceAPI2

First API is correctly generated but the second one has incorrect routes in its API Gateway (it has the routes of the API 1...) After some investigations the problem is located in chalice.cli.factory.CLIFactory.load_chalice_app() : app = importlib.import_module('app') the runtime/app.py of the first API is already loaded : calling a second time import_module() doesn't reload it and the wrong 'app.py' file is used for the second template generation.

proposed (working) workaround is to reload the module with importlib.reload() :

if "app" in sys.modules:
    app = importlib.import_module('app')
    importlib.reload(app)
else:
    app = importlib.import_module('app')

[jk2l]

+1 on this, i am doing similar thing and I am getting same issue

[lymmurrain]

Chalice causes all Chalice projects in stages after the first stage to import the last Chalice project in the first stage. This is because Chalice's load_chalice_app function adds the path to the Chalice project at the beginning of the sys path. Even if the app package is removed from sys.modules, it still imports the wrong app package. The problematic code exists in chalice/cli/factory.py.

def load_chalice_app(
        self,
        environment_variables: Optional[MutableMapping] = None,
        validate_feature_flags: Optional[bool] = True,
) -> Chalice:
    # validate_features indicates that we should validate that
    # any expiremental features used have the appropriate feature flags.
    if self.project_dir not in sys.path:
        sys.path.insert(0, self.project_dir)
    ...

Additionally, there are other packages with the same name in my Chalice projects, so simply removing app and chalicelib is not enough. My solution is as follows (modifying the load_chalice_app function):

def load_chalice_app(
        self,
        environment_variables: Optional[MutableMapping] = None,
        validate_feature_flags: Optional[bool] = True,
) -> Chalice:
    # validate_features indicates that we should validate that
    # any expiremental features used have the appropriate feature flags.
    if self.project_dir not in sys.path:
        sys.path.insert(0, self.project_dir)
    # The vendor directory has its contents copied up to the top level of
    # the deployment package. This means that imports will work in the
    # lambda function as if the vendor directory is on the python path.
    # For loading the config locally we must add the vendor directory to
    # the path so it will be treated the same as if it were running on
    # lambda.
    vendor_dir = os.path.join(self.project_dir, 'vendor')
    if os.path.isdir(vendor_dir) and vendor_dir not in sys.path:
        # This is a tradeoff we have to make for local use.
        # The common use case of vendor/ is to include
        # extension modules built for AWS Lambda.  If you're
        # running on a non-linux dev machine, then attempting
        # to import these files will raise exceptions.  As
        # a workaround, the vendor is added to the end of
        # sys.path so it's after `./lib/site-packages`.
        # This gives you a change to install the correct
        # version locally and still keep the lambda
        # specific one in vendor/
        sys.path.append(vendor_dir)
    if environment_variables is not None:
        self._environ.update(environment_variables)
    try:
        ## BEGIN Added code
        PRELOADED_MODULES = set()

        def init():
            # local imports to keep things neat
            from sys import modules
            import importlib

            nonlocal PRELOADED_MODULES

            # sys and importlib are ignored here too
            PRELOADED_MODULES = set(modules.values())

        def delete_imported():
            from sys import modules
            import importlib
            import gc
            deleted = set()
            for module in set(modules.values()) - PRELOADED_MODULES:
                try:
                    del sys.modules[module.__name__]
                    deleted.add(module.__name__)
                except:
                    # there are some problems that are swept under the rug here
                    pass
            gc.collect()
            print("Deleted modules: ", deleted)
            if self.project_dir  in sys.path:
                sys.path.remove(self.project_dir)

        init()
        ## END
        print(f'app in sys.modules: {sys.modules.get("app")}')
        app = importlib.import_module('app')
        chalice_app = getattr(app, 'app')
        ## BEGIN Added code
        delete_imported()
        ## END

By the way, if the Chalice project uses the Pydantic module, you should skip packages that use Pydantic when deleting imported modules in delete_imported. Otherwise, it may report errors such as duplicate validator functions.