gchq / Gaffer

A large-scale entity and relation database supporting aggregation of properties
Apache License 2.0
1.75k stars 353 forks source link

Move the functions (koryphe) library out of Gaffer to a dedicated repository #779

Closed t616178 closed 7 years ago

t616178 commented 7 years ago

The functions library should be moved out of Gaffer and into it's own repository so that it can easily be shared between projects.

gaffer01 commented 7 years ago

I think it's fine to have a dependency on an external functions library, but Gaffer should own the core interfaces that define the functions it can do (aggregate, filter elements, transform elements, etc) and these should not depend on external functions libraries. It must be possible to plug-in other function libraries later - we may want to write higher performance libraries later.

p013570 commented 7 years ago

@gaffer01 - I agree. I have raised a separate issue to refactor the code to remove the hard dependency on Koryphe functions: gh-820.

p013570 commented 7 years ago

As part of this issue we should also move the functions in core-library to the koryphe library.

p013570 commented 7 years ago

I've created a koryphe repository ready.

p013570 commented 7 years ago

The code has been added to the develop branch in the koryphe repo. Once koryphe version 0.1.0 has been released we can remove the koryphe code out of the Gaffer repository.

p013570 commented 7 years ago

Migration steps

A large number of functions, aggregators and transforms have now been moved into the Koryphe library. This has meant we have had to rename the packages for these functions. e.g

uk.gov.gchq.gaffer.function.filter.IsMoreThan
-> uk.gov.gchq.koryphe.impl.predicate.IsMoreThan

We have also renamed packages so they match the Java 8 function interfaces:

filter -> predicate
aggregate -> binaryoperator
transform -> function

e.g

uk.gov.gchq.gaffer.sketches.datasketches.theta.function.aggregator.UnionAggregator
-> uk.gov.gchq.gaffer.sketches.datasketches.theta.binaryoperator.UnionAggregator

Note that some fields have also been renamed to maintain a consistent naming pattern and that this will affect any serialised JSON.

e.g

 "postTransformFilterFunctions": [
        {
          "function": {
            "class": "uk.gov.gchq.gaffer.function.ExampleFilterFunction"
          }
        }
      ]

->

 "postTransformFilterFunctions": [
        {
          "predicate": {
            "class": "uk.gov.gchq.gaffer.function.ExampleFilterFunction"
          }
        }
      ]

If you have any custom predicates/binaryoperators/functions then you may need to add a dependency on the koryphe test jar so your unit tests can extend the koryphe abstract test classes:

<dependency>
    <groupId>uk.gov.gchq.koryphe</groupId>
    <artifactId>koryphe</artifactId>
    <version>0.1.0</version>
    <type>test-jar</type>
    <scope>test</scope>
</dependency>

See the predicate examples for further details.

p013570 commented 7 years ago

Merge into develop.