Defaults being applied to 1st element only when merging array of objects with arrayMergeById

[ravensorb]

I am running into an interesting issue with a complex json structure that includes a deep set of objects and arrays of objects. It appears that for every array only the 1st element has the default values defined in the schema applied to it -- every other element in the array does not. Is there something I need to do with the strategy or configuration.

Here is a link to the schema file that I am using. Is it possible i have missed something?

https://gist.github.com/ravensorb/5513ccc1b488832204498300fb868467

The specific one I am testing is "#/definitions/traefik/properties/routers"

[avian2]

Hi. I'm confused by your question. I don't think jsonmerge has any feature that applies default values like you describe.

[ravensorb]

Interesting -- so the behavior I am seeing is that if I add a "default" attribute to a property in the schema -- it is applied during the merge process. Is this not expected behavior?

Example:

"hostcontaineritem": {
            "$id": "#/definitions/hostcontaineritem",
            "type": "object",
            "default": {}, 
            "mergeStrategy": "objectMerge",
            "required": [
                "host",
                "container",
                "comment"
            ],
            "properties": {
                "host": {
                    "$id": "#/definitions/hostcontaineritem/properties/host",
                    "type": "string",
                    "default": ""
                },
                "container": {
                    "$id": "#/definitions/hostcontaineritem/properties/container",
                    "type": "string",
                    "default": ""
                },
                "comment": {
                    "$id": "#/definitions/hostcontaineritem/properties/comment",
                    "type": "string",
                    "default": "Default Comment"
                }
            },
            "additionalProperties": true
        },

if you look at "#/definitions/hostcontaineritem/properties/comment" -- the "Default Comment" value will be present in the merged result (if that property does not exist in the source document.

Here is my simple use case

with open(jsonFile1,) as fFile:
            baseJsonData = json.load(fFile) # Load a base document that is the starting point

with open(jsonFile2,) as fFile:
            changedJsonData = json.load(fFile) # Load the changes

with open(schemaFile,) as fFile:
            resultSchema = json.load(fFile) # Load the schema

mergerObj = Merger(result)

# print(baseJsonData["apps"][1]["containters"][0]["comment"]) # <-- this would error as the "comment" object is not in the baseJsonData

resultMerged = self.mergerObj.merge(copy.deepcopy(baseJsonData), changedJsonData) # Note: deepcopy was needed as baseJsonData was being updated in place (I needed a immutable copy to use multiple times)

print(baseJsonData["apps"][1]["containters"][0]["volume"][0]["comment"]) # <-- This works
print(baseJsonData["apps"][1]["containters"][1]["volume"][1]["comment"]) # <-- This throws an exception

Update: Ok, so I just looked over your code base (should have done that first) and I too am not sure where the default values are being applied ??

[avian2]

I can't imagine how this would work. As far as I know nothing in jsonmerge looks at the "default" keyword in the schema. Can you provide a small, self-contained example? https://github.com/avian2/jsonmerge#reporting-bugs-and-contributing-code

[ravensorb]

Yes, I'll create a sample -- might be later today.

[ravensorb]

I updated the gist link above with a working same using a complex object. You can see in the output where a default value is added to the 1st element in one of the arrays.

I am wondering if this might actual be an issue with the jsonschema library?

[avian2]

I see nothing in your test case that suggests that the "default" keyword is used at all. You can verify that by removing it and you should get the same result.

The result1["traefik"]["routers"][0]["priority"] you see after merge comes from baseContainerJson["traefik"]["routers"][0]["priority"].

As far as I can see, the /traefik/routers array is correctly merged using the arrayMergeById strategy. You specify in your schema that the arrays should be merged by the name property. Since no array items in your head or base document share the same name property, the two arrays are simply appended. Hence the first element of the resulting array is the same as the one in base - the one containing priority.

[ravensorb]

Sigh -- I looked at that over and over and I just plain missed that :-/ My apologies for that. I did find that there is an option to enhance the validator class to allow it to implement defaults if/when they are not found and that might be the way to go for my needs.

Thanks for taking a look and for the support -- your library was exactly what i was looking for in terms of supporting different/complex merge scenarios.