Problem encoding message using schema with array of records

[vadymrybak]

Does this library support encoding messages with schema that is array of records?

I keep getting error when trying to encode message that my message is invalid.

Schema is quiet big to insert but basically similar to this

[
    {
        "type": "record",
        "namespace": "com.wf.api.instance",
        "name": "SyntaxRef",
        "doc": [
            "Reference to a syntax element"
        ],
        "fields": [
            {
                "name": "syntaxId",
                "type": "int",
                "doc": "Syntax id"
            }
        ]
    },
    {
        "type": "record",
        "namespace": "com.wf.api.task",
        "name": "WfControlKey",
        "doc": [
            "Definition of control key",
            "Created based on TC definition XML."
        ],
        "fields": [
            {
                "name": "rootInstanceKey",
                "type": [
                    "null",
                    "string"
                ],
                "default": null
            },
            {
                "name": "instanceKey",
                "type": [
                    "null",
                    "string"
                ],
                "default": null
            },
            {
                "name": "controlKey",
                "type": [
                    "null",
                    "string"
                ],
                "default": null
            }
        ]
    }
]

[bencebalogh]

Sounds like more of an issue for avsc, but I don't think this schema is valid. I've not checked the documentation right now, so I could remember this wrong, but I think your schema should look like this something as:

{
    "name": "test_schema".
    "type": "array",
    "items": [
        <record here with 2 fields as SyntaxRef and WfControlKey records>
     ]
}

[bencebalogh]

Going to close this for now, let me know if you have any questions or if there's anything I could help with.

[vadymrybak]

I think I figured this one out. The problem is with how avsc is handling array of records. I did similar test with just their library and it was throwing same error. I opened a ticket with them below:

https://github.com/mtth/avsc/issues/283

So after this ticket solved my issue I switched back to this library. The only way I got it to work is like below:

const schemaRegistry: string = 'http://localhost:8096';
export const registry: ISchemaRegistry = avroSchemaRegistry(schemaRegistry);

const schema: string = fs.readFileSync('../avro/wf-instance.avsc', {encoding: 'utf8'});
const message: object = {
    "controlKey": {
        "rootInstanceKey": "bd1f8011-9341-46d2-bb3b-c88c144cd1",
        "instanceKey": "bd1f8011-9341-46d2-bb3b-c8s8cd144cd0",
        "controlKey": "bd1f8011-9341-46d2-bb3b-c88cd1a49sc2d0"
    },
    "taskTypeNm": "add_sub",
};

const _registry: any = {};
const _schema: any      = JSON.parse(schema);
avsc.Type.forSchema(_schema, {registry: _registry});
const usedType: any = _registry['com.vad.wf.api.task.WfTask'];

registry.encodeMessage('wf-core-engine-wf-exec-ea-request', usedType, message, {})
    .then((msg: Buffer) => {
        console.log("ENCODED MESSAGE:");
        console.log(msg);
    })
    .catch( (err: object) => {
        console.log("ERROR encodeMessage!");
        console.log(err);
    });

I also simplified schema to have only two records:

[
    {
        "type": "record",
        "namespace": "com.vad.wf.api.task",
        "name": "WfControlKey",
        "doc": [
            "Definition of control key",
            "Created based on TC definition XML."
        ],
        "fields": [
            {
                "name": "rootInstanceKey",
                "type": [
                    "null",
                    "string"
                ],
                "default": null
            },
            {
                "name": "instanceKey",
                "type": [
                    "null",
                    "string"
                ],
                "default": null
            },
            {
                "name": "controlKey",
                "type": [
                    "null",
                    "string"
                ],
                "default": null
            }
        ]
    },
    {
        "type": "record",
        "namespace": "com.vad.wf.api.task",
        "name": "WfTask",
        "fields": [
            {
                "name": "controlKey",
                "type": "com.vad.wf.api.task.WfControlKey"
            },
            {
                "name": "taskTypeNm",
                "type": [
                    "null",
                    "string"
                ],
                "doc": "Task type name"
            }
        ]
    }
]

The 'magic' is happening in this line of code:

const usedType: any = _registry['com.vad.wf.api.task.WfTask'];

Instead of passing the whole schema to the library - we are passing only one record object from the registry. But then we need to call now 2 libraries instead of just this one.

Do you think there is something you guys could improve? Maybe add additional parameter - which record object to use from registry?

Thanks!