aws-cloudformation / cloudformation-cli

The CloudFormation Provider Development Toolkit allows you to author your own resource providers and modules that can be used by CloudFormation.
Apache License 2.0
318 stars 161 forks source link

FlatteningError while working with remote "$ref" in typeConfiguration #1084

Open Scribbd opened 3 months ago

Scribbd commented 3 months ago

I am working on a set of custom resource types that share two pieces of configuration over all planned types:

To limit the amount of duplicate JSON definitions, I took a page out of the teams book and created a bucket where I placed my remote definitions. And uploaded the shared definitions in that bucket.

Resulting in a definition JSON like follows:

{
    "$schema": "https://schema.cloudformation.us-east-1.amazonaws.com/provider.definition.schema.v1.json#",
    "properties": {
        "Title": {
            "description": "Title/Name of the folder.",
            "type": "string"
        },
        "Environment": {
            "$ref": "https://s3.eu-west-1.amazonaws.com/my.fake.schemas.bucket/environment.def.v0.json"
        }
    },
...
   "typeConfiguration": {
        "additionalProperties": false,
        "properties": {
            "EnvironmentMapping":{
                "description": "A list of environment definitions detailing which secret is bound to which host.",
                "type":"array",
                "insertionOrder": false,
                "items": {
                    "$ref": "https://s3.eu-west-1.amazonaws.com/my.fake.schemas.bucket/environment-config.def.v0.json"
                }
            }
        },
        "required": ["EnvironmentConfiguration"]
    }
}

It works for the Environment property. That by itself and cfn generate finishes without problem. But when I add the reference to the typeConfiguration, cfn generate will produce the following error: rpdk.core.jsonutils.utils.FlatteningError: Invalid ref: ('remote', 'schema1')

When perusing through the logs, I do see both schema's being picked up and verified. Both written to remote/schema0 and remote/schema1. And files are generated, but the config JSON looks as follows:

{
    "additionalProperties": false,
    "properties": {
        "EnvironmentConfiguration": {
            "type": "array",
            "insertionOrder": false,
            "items": {
                "$ref": "#/remote/schema1"
            }
        }
    },
    "required": [
        "EnvironmentConfiguration"
    ],
    "definitions": {},
    "typeName": "ORG::App::Folder"
}

I tried a few things:

I might dive into the code and see how both blocks are rendered and how they differ. But it is late for me, at the time of writing this issue, and maybe someone will see an obvious mistake I missed.

Thanks in advance.

Scribbd commented 3 months ago

There is a pragmatic fix:

diff --git a/src/rpdk/core/project.py b/src/rpdk/core/project.py
index bbf0922..b0870ae 100644
--- a/src/rpdk/core/project.py
+++ b/src/rpdk/core/project.py
@@ -509,6 +509,7 @@ class Project:  # pylint: disable=too-many-instance-attributes,too-many-public-m
         if "typeConfiguration" in self.schema:
             configuration_schema = self.schema["typeConfiguration"]
             configuration_schema["definitions"] = self.schema.get("definitions", {})
+            configuration_schema["remote"] = self.schema.get("remote", {})
             configuration_schema["typeName"] = self.type_name
             self.configuration_schema = configuration_schema

By simply adding the remote definitions to the to-be generated type-configuration. It will pass the flattening check, and still be valid according to the meta-schema which allows for a remote key.

I was unable to find a meta-schema specific for the type-configuration. So, I am assuming that the provider one applies here as well.

Scribbd commented 2 months ago

I found the appropriate schema for type configuration: https://github.com/aws-cloudformation/cloudformation-cli/blob/master/src/rpdk/core/data/schema/provider.configuration.definition.schema.v1.json This one does not allow for the 'remote'-attribute. I will be closing the pull request as my 'pragmatic' fix won't be compliant.