Unable to use `properties:` of `type: object`

[tarilabs]

I've partially discussed this with @lburgazzoli, currently it's not possible to make use of $subject as the GoLang would serialize any yaml in its GoLang string format --iiuc. /cc @valdar @danielezonca

I have a Kamelet defined as:

apiVersion: camel.apache.org/v1alpha1
kind: Kamelet
metadata:
  name: mmk
  labels:
    camel.apache.org/kamelet.type: "action"
spec:
  definition:
    title: "mmk"
    description: prova
    properties:
      kdtable:
        title: prova
        description: prova
        type: object
  dependencies:
  - "camel:core"
  - "camel:kamelet"
  - "github:kiegroup.yard/yard-kdtable/camelk-SNAPSHOT"
  template:
    beans:
      - name: mmbean
        type: '#class:org.drools.yard.kdtable.KdtableProcessor'
        properties:
          kdtable: '{{kdtable}}'
    from:
      uri: "kamelet:source"
      steps:
      - process:
          ref: '{{mmbean}}'
      - to: "log:info"

I want to use the property kdtable in the Kamelet binding as a YAML. Currently the only workaround is to supply its value as a string [block] in order not to be passing through some serialization of the GoLang:

apiVersion: camel.apache.org/v1alpha1
kind: KameletBinding
metadata:
  name: timer-source-binding
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: timer-source
    properties:
      message: hello world
      period: 5000
  steps:
    - ref:
        kind: Kamelet
        apiVersion: camel.apache.org/v1alpha1
        name: mmk
      properties:
        # set as block string (undesirable, but the only available current workaround)
        kdtable: |
          type: DecisionTable
          inputs: ['Age', 'Previous incidents?']
          rules:
          - when: ['<21', false]
            then: 800
          - when: ['<21', true]
            then: 1000
          - when: ['>=21', false]
            then: 500
          - when: ['>=21', true]
            then: 600
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1alpha1
      name: mmk
    properties:
      # set as yaml, unfortunately it is not deserialized as a JSONNode or Map in Java, it is serialized as string by GoLang --iiuc
      kdtable:
          type: DecisionTable
          inputs: ['Age', 'Previous incidents?']
          rules:
          - when: ['<21', false]
            then: 800
          - when: ['<21', true]
            then: 1000
          - when: ['>=21', false]
            then: 500
          - when: ['>=21', true]
            then: 600

My processor is defined as:

public class KdtableProcessor implements Processor {

    private Object kdtable;

    @Override
    public void process(Exchange exchange) throws Exception {
        LOG.info("this processor/bean field 'kdtable' was set to: {}", kdtable);
        LOG.info("this processor/bean field 'kdtable' class is: {}", kdtable.getClass());

//... and getters, setters, etc.

The output of the above binding is:

2023-01-16 15:38:22,557 INFO  [org.dro.yar.kdt.KdtableProcessor] (Camel (camel-1) thread #1 - timer://tick) this processor/bean field 'kdtable' was set to: type: DecisionTable
inputs: ['Age', 'Previous incidents?']
rules:
- when: ['<21', false]
  then: 800
- when: ['<21', true]
  then: 1000
- when: ['>=21', false]
  then: 500
- when: ['>=21', true]
  then: 600
2023-01-16 15:38:22,558 INFO  [org.dro.yar.kdt.KdtableProcessor] (Camel (camel-1) thread #1 - timer://tick) this processor/bean field 'kdtable' class is: class java.lang.String

...

2023-01-16 15:38:22,562 INFO  [org.dro.yar.kdt.KdtableProcessor] (Camel (camel-1) thread #1 - timer://tick) this processor/bean field 'kdtable' was set to: map[inputs:[Age Previous incidents?] rules:[map[then:800 when:[<21 false]] map[then:1000 when:[<21 true]] map[then:500 when:[>=21 false]] map[then:600 when:[>=21 true]]] type:DecisionTable]
2023-01-16 15:38:22,562 INFO  [org.dro.yar.kdt.KdtableProcessor] (Camel (camel-1) thread #1 - timer://tick) this processor/bean field 'kdtable' class is: class java.lang.String

The second variation in the kamelet binding (supplying kdtable as yaml, not as a string-block) is what I want to use.

• I would prefer to be deserialized in Java as a Jackson JSONNode or as a java.lang.Map, of the yaml originally supplied
• if the above is not possible, I do not want the map[ ... ] serialization on the Java side in the string, and to maintain the new line and formatting with indentation of whitespace or tabs (so to use manually Jackson myself).

Can you kindly consider amending Camel-K to support this use-case, please? Thanks 🙏

[squakez]

I'm marking this as a feature request for now. I don't have a full understanding of the impact, it will need to be analyzed more in detail. I have a vague memory of @astefanutti being involved in the past in some serialization experiments around it.

cc @oscerd as it would impact eventually the Kamelet spec definition.

[lburgazzoli]

I had a brief look at how things works as today and so far what I found is that essentially we lack a way to handle properties of type object.

As today camel-k translates each kamelet property in a java property like:

kamelt.kamelet.${kameelt-id}.${id}.${property} = ${value}

So @tarilabs workaround works because if you define a property as a string, then it fall into a known path. However if a property is an object, that would fail.

A possible solution would be to:

• serialize objects as json + base64
• leverage camel's type converters for de-serialize the json to something meaningful

So a property would then look like:

kamelt.kamelet.mmk.${id}.ktable = {{base64:eyAidHlwZSI6ICJEZWNpc2lvblRhYmxlIiB9}}

Is up to the kamelet developer to provide a a type converter, something like:

@Converter(generateLoader = true)
public class KieConverter {
    @Converter
    public static DecisionTable toDecisionTable(String raw)  {
        return  new ObjectMapper().readValue(), DecisionTable.class);
    }
}

At some point we may be able to offer some automatic conversion but as first step we can delegate some of the conversion logic to the kamelet developer.

@squakez @tarilabs what do you think ?

[squakez]

I think @christophd is working on some data type configuration, maybe this is overlapping some areas he is touching as well.

[tarilabs]

am I understanding correctly base64 is basically to circumvent the .properties limitation on newlines/whitespace/indentation, and basically guarantee the JSON representation (string) can be placed on 1-single line (for the .properties)?

If that understanding is correct, is fine with me with base64 or other encoding, and granted, is up to the consumer of that configuration item to be aware it's a JSON representation which need decoding in some way (which is typically the case, as a developer accessing a configuration parameter). At that point the developer could decode using Jackson by providing a marshalling class when known, or decode using Jackson by unmarshalling to generic JSONNode or Map.

[lburgazzoli]

am I understanding correctly base64 is basically to circumvent the .properties limitation on newlines/whitespace/indentation, and basically guarantee the JSON representation (string) can be placed on 1-single line (for the .properties)?

If that understanding is correct, is fine with me with base64 or other encoding, and granted, is up to the consumer of that configuration item to be aware it's a JSON representation which need decoding in some way (which is typically the case, as a developer accessing a configuration parameter). At that point the developer could decode using Jackson by providing a marshalling class when known, or decode using Jackson by unmarshalling to generic JSONNode or Map.

No, camel-k should do the necessary configuration magic to ensure the user does not need to worry about base64 encoding. What the user's bean would receive is either a plain string/bytes or a decoded object if a related type converter is provided.

[tarilabs]

@lburgazzoli

may I deduce the actual snippet you intended was:

@Converter(generateLoader = true)
public class IOConverter {
    @Converter

-    public static DecisionTable toDecisionTable(Exchange exchange, String raw)  {
-        return  new ObjectMapper().readValue(), DecisionTable.class);
+    public static DecisionTable toDecisionTable(String raw)  {
+        return  new ObjectMapper().readValue(raw, DecisionTable.class);

as in, not being a type converter for exchange-parameter but Processor property (of the instance level, not Exchange-level), please?

[lburgazzoli]

yes correct, the exchange in this case does not make any sense, example updated

[tarilabs]

Thanks for the confirmation! That should work with me 👍

[github-actions[bot]]

This issue has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!

[tarilabs]

Bump