apache / plc4x-extras

PLC4X The Industrial IoT adapter
https://plc4x.apache.org/
9 stars 6 forks source link

[Bug]: NiFi integration - Tag names with dots throws Avro exception even when not using Avro #195

Open QuanticPony opened 1 month ago

QuanticPony commented 1 month ago

What happened?

Reproduction steps:

  1. Using the Plc4xSourceRecordProcessor set the writer (processor output) to json, csv or any other than avro.
  2. When declaring a tag set the name as A.1 or 1.A with any address and driver.
  3. Run A org.apache.avro.SchemaParseException is raised.

On the mailing list @ottobackwards correctly pointed out that using the avro specification (https://avro.apache.org/docs/1.7.7/spec.html#Names) for tag names does not raise the exception.

This is due to creating an avro schema from the response and converting it to a NiFi schema before using the writer configured on the processor. That step is is not needed if the processor creates a NiFi record directly. That would allow to have tags not allowed on avro specification for field names on other writers and would not be a problem for avro as it's writer handles non valid field names (https://github.com/apache/nifi/blob/35219052e13c9a2a2f3a40e125832a872067d1b1/nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java#L132).

Version

0.13.0-SNAPSHOT

Programming Languages

Protocols