[KIEKER-1040] Document/Develop pattern how to extend existing record types

[rju]

JIRA Issue: KIEKER-1040 Document/Develop pattern how to extend existing record types Original Reporter: Andre van Hoorn

---

[rju]

author André van Hoorn -- Mon, 21 Oct 2013 16:22:10 +0200

Examples can be found in flow records. Pattern can be reconstructed/documented based on this work.

[rju]

author Jan Waller -- Tue, 17 Jun 2014 14:38:52 +0200

Reiner Jung: 1.10?

[rju]

author rju -- Thu, 19 Jun 2014 10:43:14 +0200

As we can generate the record with the IRL generator, we should explain how to use the IRL and provide a precise description of the patterns used in the generator and Kieker for future development.

[rju]

author rju -- Thu, 26 Jun 2014 13:37:09 +0200

There will be a presentation on 2nd July 2014 10:00

[rju]

author André van Hoorn -- Fri, 4 Jul 2014 09:49:28 +0200

See also KIEKER-926 Done

[rju]

author Thomas F. Düllmann -- Mon, 7 Jul 2014 20:02:35 +0200

When I tried to let a record (ExtendedStorableDetectionResult) inherit from an existing one (StorableDetectionResult), I ran into a test failure in TestRecordSerialization which stated, that I had passed too many objects to the constructor.

The cause for this was, that the constructor passes the object array (containing all parameters as Object) to the parent class constructor which checks (AbstractMonitoringRecord.checkArray(..)) the number of items in the array with its own number of required parameters. This obviously leads to a problem, as I added another attribute in the inheriting class.

Compared with another inheriting record class (CompilationRecord inheriting from AbstractJVMRecord) the check in the superclass is not done in AbstractJVMRecord but is not done in the StorableDetectionResult. This leads to the failing when testing in the ExtendedStorableDetectionResult.
For now I implemented a workaround for this failing test (passing a subset of the object array), but it might be better if there would be one way to do it consistently (either with or without the check in the superclass).

[rju]

author nils-christian -- Mon, 7 Jul 2014 20:08:14 +0200

Once we use the record description language, this should no longer be an issue (or an issue that has to be corrected in the generator).

[rju]

author rju -- Mon, 7 Jul 2014 20:27:07 +0200

You might want to try to generate the class with that record language (instrumentation record language (IRL)). As this looks like a misinterpretation of the record API. However, I am not certain, as I cannot see your implementation. I suppose your constructor looks like this:

public ExtendedStorableDetectionResult(final Object[] values) { // NOPMD (direct store of values)
                AbstractMonitoringRecord.checkArray(values, StorableDetectionResult.TYPES);

or your TYPES declaration is identical to that of StorableDetectionResult.TYPES. You need to define a new TYPES array and you have to fix the size parameter.

BTW: in which branch are you working?

[rju]

author Thomas F. Düllmann -- Tue, 8 Jul 2014 13:29:31 +0200

I work on the opad-tslib-integration branch.
The point is, that the (previously existing) class StorableDetectionResult implemented the constructor different in comparison to another (previously existing) class AbstractJVMRecord. So I ran into test failures because of the different implementation in the parent classes.
AbstractJVMRecord

public AbstractJVMRecord(final Object[] values) { // NOPMD (direct store of values)
    this.timestamp = (Long) values[0];
    this.hostname = (String) values[1];
    this.vmName = (String) values[2];
}

CompilationRecord extends AbstractJVMRecord:

public CompilationRecord(final Object[] values) { // NOPMD (direct store of values)
    super(values);
    AbstractMonitoringRecord.checkArray(values, TYPES);

    this.jitCompilerName = (String) values[3];
    this.totalCompilationTimeMS = (Long) values[4];
}

StorableDetectionResult:

public StorableDetectionResult(final Object[] values) { // NOPMD (direct store of values)
    AbstractMonitoringRecord.checkArray(values, StorableDetectionResult.TYPES);

    this.applicationName = (String) values[0];
    this.value = (Double) values[1];
    this.timestamp = (Long) values[2];
    this.forecast = (Double) values[3];
    this.score = (Double) values[4];
}

ExtendedStorableDetectionResult extends StorableDetectionResult:

public ExtendedStorableDetectionResult(final Object[] values) { // NOPMD
    super(Arrays.copyOfRange(values, 0, values.length - 1));
    AbstractMonitoringRecord.checkArray(values, ExtendedStorableDetectionResult.TYPES);

    this.anomalyThreshold = (Double) values[5];
}

As you can see, the AbstractJVMRecord does not do the array check, which allows the inheriting class to pass even bigger parameters than the parent class expects for itself. In the ExtendedStorableDetectionResult I had to reduce the passed array to the expected size of the parent class.

[rju]

author rju -- Mon, 28 Jul 2014 10:47:50 +0200

The user guide should use the IRL compiler and tooling (optional). The development documentation should mention additional info on the record structure.

[rju]

author André van Hoorn -- Thu, 4 Sep 2014 16:53:02 +0200

Add link to IRL compiler documentation.

[rju]

author rju -- Fri, 4 Sep 2015 11:24:15 +0200

The present IRL generator (and also past generators) produce code for array types in two different ways. It supports bounded arrays, e.g., int[10] property, and unbounded array, e.g., int[] property. And the initialization with values.

The code generator produces for:

package demo

author 'Reiner Jung' 
entity ArrayExample {
    int[10] staticArray
    int[] dynamicArray  
}

The following Java class (in fragments, without deprecated parts.):

public class ArrayExample extends AbstractMonitoringRecord implements IMonitoringRecord.Factory, IMonitoringRecord.BinaryFactory {
    /** Descriptive definition of the serialization size of the record. */
    public static final int SIZE = TYPE_SIZE_INT // ArrayExample.staticArray
             + TYPE_SIZE_INT // ArrayExample.dynamicArray
    ;
    private static final long serialVersionUID = -2048572783965064715L;

    public static final Class<?>[] TYPES = {
        int[].class, // ArrayExample.staticArray
        int[].class, // ArrayExample.dynamicArray
    };

    /* user-defined constants */
    /* default constants */
    /* property declarations */
    private final int[] staticArray;
    private final int[] dynamicArray;

    /**
     * Creates a new instance of this class using the given parameters.
     * 
     * param staticArray
     *            staticArray
     * param dynamicArray
     *            dynamicArray
     */
    public ArrayExample(final int[] staticArray, final int[] dynamicArray) {
        this.staticArray = staticArray;
        this.dynamicArray = dynamicArray;
    }

    /**
     * This constructor converts the given array into a record.
     * It is recommended to use the array which is the result of a call to {link #toArray()}.
     * 
     * param values
     *            The values for the record.
     */
    public ArrayExample(final Object[] values) { // NOPMD (direct store of values)
        AbstractMonitoringRecord.checkArray(values, TYPES);
        this.staticArray = (int[]) values[0];
        this.dynamicArray = (int[]) values[1];
    }

    /**
     * This constructor uses the given array to initialize the fields of this record.
     * 
     * param values
     *            The values for the record.
     * param valueTypes
     *            The types of the elements in the first array.
     */
    protected ArrayExample(final Object[] values, final Class<?>[] valueTypes) { // NOPMD (values stored directly)
        AbstractMonitoringRecord.checkArray(values, valueTypes);
        this.staticArray = (int[]) values[0];
        this.dynamicArray = (int[]) values[1];
    }

Initialization of array at construction.

   /**
     * This constructor converts the given array into a record.
     * 
     * param buffer
     *            The bytes for the record.
     * 
     * throws BufferUnderflowException
     *             if buffer not sufficient
     */
    public ArrayExample(final ByteBuffer buffer, final IRegistry<String> stringRegistry) throws BufferUnderflowException {
        // load array sizes
        this.staticArray = new int[10];
        for (int i0=0;i0<10;i0++)
            this.staticArray[i0] = buffer.getInt();

        // load array sizes
        int _dynamicArray_size0 = buffer.getInt();
        this.dynamicArray = new int[_dynamicArray_size0];
        for (int i0=0;i0<_dynamicArray_size0;i0++)
            this.dynamicArray[i0] = buffer.getInt();
    }

    /**
     * {inheritDoc}
     */
    Override
    public Object[] toArray() {
        return new Object[] {
            this.getStaticArray(),
            this.getDynamicArray()
        };
    }

    /**
     * {inheritDoc}
     */
    Override
    public void registerStrings(final IRegistry<String> stringRegistry) { // NOPMD (generated code)
    }

Writing an record with arrays. First, static array. Second, dynamic array. Dynamic arrays are slower.

   /**
     * {inheritDoc}
     */
    Override
    public void writeBytes(final ByteBuffer buffer, final IRegistry<String> stringRegistry) throws BufferOverflowException {
        // store array sizes
        for (int i0=0;i0<10;i0++)
            buffer.putInt(this.getStaticArray()[i0]);

        // store array sizes
        int _dynamicArray_size0 = this.getDynamicArray().length;
        buffer.putInt(_dynamicArray_size0);
        for (int i0=0;i0<_dynamicArray_size0;i0++)
            buffer.putInt(this.getDynamicArray()[i0]);
    }

    /**
     * {inheritDoc}
     */
    Override
    public Class<?>[] getValueTypes() {
        return TYPES; // NOPMD
    }

    /**
     * {inheritDoc}
     */
    Override
    public int getSize() {
        return SIZE;
    }

    public final int[] getStaticArray() {
        return this.staticArray;
    }

    public final int[] getDynamicArray() {
        return this.dynamicArray;
    }

}

We should discuss if this should be an official feature, or if this feature needs more work.

[rju]

author rju -- Tue, 12 Sep 2017 14:53:01 +0200

This feature is already part of the generator.

[rju]

author rju -- Tue, 12 Sep 2017 14:54:15 +0200

The language addresses this issue to some extend. It does not support an associative array. We could do that with enumerations and implicit enumerations. However, that would be a new feature request. Does anybody need that?