asnare
commented
6 years ago This is unfortunate; thanks for the report.
I don't think downgrading is the right solution, especially since I'd like to support some of the 1.8 features like the logical timestamp types. Instead I'd like to see us work around the issue (with test cases).
Some things the test-cases need to check:
- Mapping directly.
- Mapping via the jsonpath custom extractor path.
- If the value being mapped doesn't match a known value, how that's handled.
For invalid values I think it should be treated as an empty value instead of throwing an exception.
aspruds
It seems that enums are not working in the latest (1.8.2) version of Avro which is bundled with divolte-collector-0.7.0. When collecting data defined as enum, Divolte Collector throws the following exception: org.apache.avro.AvroTypeException: Not an enum: OW
Additional information is available at https://issues.apache.org/jira/browse/AVRO-1810. One possible solution is to downgrade Avro to 1.7.7.
For the reference, I'm using the following Avro field definition:
{ "name": "search", "type": [ "null", { "name": "SearchRecord", "type": "record", "fields": [ { "name": "screenName", "type": "string", "doc": "Screen name (e.g. fb_avail)" }, { "name": "language", "type": "string", "doc": "Language code in ISO 639-1 (lv, et)" }, { "name": "pointOfSale", "type": "string", "doc": "Point of Sale, for example, LV" }, { "name": "origin", "type": "string", "doc": "Origin, e.g. RIX" }, { "name": "destination", "type": "string", "doc": "Destination, e.g. TLL" }, { "name": "tripType", "type": {"name": "TripType", "type": "enum", "symbols" : ["OW", "RT"]}, "doc": "RT for round-trip, OW for one-way" }, { "name": "outboundDate", "type": "string", "doc": "Outbound date in format 2018-04-29" }, { "name": "inboundDate", "type": ["null", "string"], "default": null, "doc": "Inbound date in format 2018-04-29" }, { "name": "adults", "type": "int", "doc": "Number of adults" }, { "name": "children", "type": ["null", "int"], "default": null, "doc": "Number of children" }, { "name": "infants", "type": ["null", "int"], "default": null, "doc": "Number of infants" } ] } ], "default": null }