atlasmap-attic / atlasmap-runtime

AtlasMap runtime engine and design time services
http://atlasmap.io
Other
3 stars 8 forks source link

FieldActions should take in a list or array of FieldType #216

Closed cunningt closed 6 years ago

cunningt commented 6 years ago

Some FieldActions should apply to multiple FieldTypes - for example, DayOfWeek could apply to:

Date DateTime DateTZ DateTimeTZ Number (convert from 1-7 to Monday-Sunday)

In order for it to apply to non-hierarchical FieldTypes like Number and Date, we may need to pass a list or array of FieldType in as the source type and then do an instanceof check to see what format the input is in before processing it.

mattrpav commented 6 years ago

TL;DR; Use separate functions

Begin Long Read..

I believe having FieldActions support multiple type hierarchies and mixed-modes would go against the original design principals. To me, the proposed approach feels like an issue of taxonomy collision and/or incongruent rationalization. Using one Field Action function for those input types is efficient for the implementation of the Field Action at the expense of features to UI.

Additionally, this looks like an edge case (< 10%) of Field Actions, vs the core use case (>90%) where only single type needs to be supported. Also, consider that almost becomes impossible to reasonably support a Collection of multiple field types--- everything would eventually reduce down Object and List .. and we lose all the value in auto-type detection, auto-type conversion and the ability to provide design time hints during validation for the user in the UI.

Observations:

  1. Integer-to-Day of Week is a very different transformation from DateTime-to-Day of week. Any date-related concrete class will always be able to succeed in execution. Integer has a range concern, and an index concern (0 = Sunday or 1 = Sunday.. or 0 = Monday, etc?) , so there is possibility if incorrect inbound data.

  2. Likewise, String-to-Day of week is very different from Integer-to-Day of Week. Is the string input a string representation of the number (index and range concern)? Or is the input an abbreviation? ("M" -> Monday, or "Tue.". "Tu", "Tues", "Tues." -> Tuesday)? How to handle all the variations of abbreviations? This Field Action could have its own set of standard "Abbreviation Policies" that would need to be provided as input from the user at design time. The abbreviation policies would not apply for Integer or Date classes as input.

  3. Additionally, what if the input is an ENUM? The enum would first need to be transformed to something in one of the 3 core type hierarchies, then the corresponding Field Action can be applied.

  4. Given the core differences in the 4 scenarios, it would make sense to have 3 separate functions so the UI can properly validate the mapping, provide hints for type conversion, or prompt the need to additional input to solve the day-of-week index issue (needed for String and Integer inputs).

Proposed alternate solution:

Implement overloaded handlers (or differently named field actions) to support the various field types, and then the design time hints and runtime auto-type conversion can be applied-- all while still being able to support collections. The UI should be able to auto-align to what ever the expected input type for the mappings in question.

public String DayOfWeek(DateTZ input) (for all concrete DateTime, Date, Calendar, etc. input classes) public String DayOfWeek(Integer input, Integer index.. ) public String DayOfWeek(String input, Integer index.. )

--or--

public String DayOfWeekDate(DateTZ input, ) (for all concrete DateTime, Date, Calendar, etc. input classes) public String DayOfWeekInteger(Integer input, Integer index..) public String DayOfWeekString(String input, Integer index..)

--Rationalization--

Key Guiding Principals for AtlasMap

  1. Design Time usability has priority over runtime engine-- effectively: the engine should do extra lifting so end-users are not impacted

  2. Mapping definition files should have a long window of compatibility. A goal of AtlasMap is that the mapping definition file should last a long time (since high level formats and generic actions do not change very frequently). Forward compatibility should be achievable given the relatively static nature of this domain. Things like Java 8 having new date time field types are supported by the 'AtlasMap FieldType' for 'Date' supporting concrete Java class implementations such as Date, Calendar, GregorianCalendar, etc. The AtlasMap FieldType <-> Java class name should have reasonable abstraction and forward compatibility. New Java classes can be supported with new versions of AtlasMap w/o breaking previously defined AtlasMaps.

  3. In order to support #2.. FieldActions must "stay in their swim lane". The swim lane in this case is the corresponding Atlas FieldType Hierarchy. It is reasonable to go up/down a type hierarchy. The type hierarchies.. Enum, String, Number and DateTime. Auto Conversion should "own" the ability of crossing type hierarchies.

  4. Given #1, #2 and #3 .. for this use case we should implement separate functions.

A quick primer on "Atlas FieldTypes".. they are meant to be interpreted as 'generic' and cross-format (even though they really look tightly aligned to Java).

For example.. the Number hierarchy: 2017-atlasmap-number-fieldtype-hierarchy

For example, we don't want to treat Java's classes for AtlasMap FieldType.Integer "int" and "Integer" as different field types in Atlas mappings, field actions or conversions. For each "AtlasMap FieldType", there is a list of concrete Java class names that align.

ie. AtlasMap FieldType.Integer => int, Integer.class.. etc

and .. the DateTime hierarchy: 2017-atlasmap-datetime-fieldtype-hierarchy

ie. AtlasMap FieldType.Date => Date, Calendar, GregorianCalendar, XMLGregorianCalendar, etc

This allows AtlasMap runtime to support add'l class types w/o having to redefine maps or FieldActions. For example, if Joda classes need to be supported, simply register those class names as implementations for a given AtlasMap FieldType.

For a Field Action to support Long and Integer, the Field Action should register itself as high up the tree as it needs. This allows a wide-range of compatibility of concrete field types and the auto-conversion service will handle the rest. Likewise, to have narrow support, register as low down the tree to lock in a specific type.

igarashitm commented 6 years ago

merged

mattrpav commented 6 years ago

Notes on solution:

  1. The AtlasMap runtime now supports FieldActions with overloaded names
  2. This is implemented using the 'displayName' field on the FieldAction as defined in the atlas-model project. Previously, AtlasMap was using .class.getSimpleName() by convention. The default is still to use class.getSimpleName() via the io.atlasmap.v2.FieldAction interface via default method.
  3. The DefaultAtlasFieldActionService has been updated to reflect these changes, as well as fix a correctness issue where the return type was not correct if the FieldAction sourceType and targetType where not the same (see: #260)

To add an overloaded Field Action:

  1. Update the atlas-model via atlas-actions-v2.xsd (or create a new module)
  2. Add a 'displayName' field with a constant value to override the default method
  3. Create the FieldAction implementation(s) and set the name='' annotation to match the 'displayName' value from the model class.

I created the atlas-itests-mock-fieldactions class as a demonstration