Camel connector should have a way to declare an option as part of a global configuration

[davsclaus]

In the camel-connector.json file https://github.com/redhat-ipaas/connectors/blob/master/connectors/twitter-mention-connector/src/main/resources/camel-connector.json

We can add a new entry to specify which options are part of global configuration

"globalOptions" : [ "accessToken", "accessTokenSecret", "consumerKey", "consumerSecret" ],

And for the salesforce it would be https://github.com/redhat-ipaas/connectors/blob/master/connectors/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json

"globalOptions" : [ "loginUrl", "clientId", "clientSecret", "refreshToken" ],

These global options are most likely global options for authentication details, but they would not have to be limited to that.

We could also enrich this information in the Camel component schema file which is generated based on the camel-connector json https://github.com/redhat-ipaas/connectors/blob/master/connectors/salesforce-upsert-contact-connector/src/main/resources/camel-connector-schema.json

And we could add an entry if an option is global or not in the other schema file that has fine grained details of every options.

    global: true | false

For each row, eg

 "loginUrl": { "kind": "property", "displayName": "Login Url", global: true, "group": "security", "label": "security", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": false, "defaultValue": "https://login.salesforce.com", "description": "Salesforce login URL defaults to https://login.salesforce.com" },
    "clientId": { "kind": "property", "displayName": "Client Id", global: true, "group": "security", "label": "security", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": true, "description": "Salesforce connected application Consumer Key" },
    "clientSecret": { "kind": "property", "displayName": "Client Secret", global: true, "group": "security", "label": "security", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": true, "description": "Salesforce connected application Consumer Secret" },
    "refreshToken": { "kind": "property", "displayName": "Refresh Token", global: true, "group": "security", "label": "security", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": true, "description": "Salesforce connected application Consumer token" }
  },
  "properties": {
    "sObjectIdName": { "kind": "parameter", "displayName": "SObject Id Name", global: false, "group": "common", "required": true, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": false, "description": "SObject external ID field name" },
    "sObjectIdValue": { "kind": "parameter", "displayName": "SObject Id Value", global: false, "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "secret": false, "description": "SObject external ID field value" }
  }
}

However this requires a little bigger change in Camel as we add a new option in the json format.

[davsclaus]

@jimmidyson @rhuss @KurtStam @gashcrumb - I just created a quick ticket about that global true|false information we talked about today that would make lives of iPaaS easier.

I suggest to look at the salesforce-upsert Camel connector as an example and the 2 json files it includes: https://github.com/redhat-ipaas/connectors/tree/master/connectors/salesforce-upsert-contact-connector/src/main/resources

Those 2 json files is what we have as information about the connector that the iPaaS will have available. From the connector catalog backend when we get that implemented.

And then at this ticket and see if what I wrote in the top description makes sense for the iPaaS.

[gashcrumb]

Yeah, even just the globalOptions outside of the form config would be fine, I can take that into account when parsing the form config easily enough.

Hey, unrelated question, what's properties again?

[davsclaus]

Okay thanks.

So properties is the first name we chose for these json files in the camel-catalog for regular Camel Components which we started on in Camel 2.15.

It is really endpointOptions. Likewise there is componentProperties which is really componentOptions.

But we are stuck with these names as all the tooling etc expect that name, eg fuse ide and others.

[gashcrumb]

That's fine. Are these properties then also part of the connection configuration, i.e. should I also be taking them into account when generating the form I wonder...

[KurtStam]

@davsclaus , alternatively we could do a comparison on property name to filter out the common ones? And present those in the UI for the expert developer to mark them as 'global'? It might be risky, but something to think about?

[davsclaus]

So in the camel-connector.json file you have two places where the possible options the connector allows to be configured from the iPaaS.

They are listed as names in the arrays as: componentOptions and endpointOptions. And a connector can have either only componentOptions or endpointOptions or both of them. Its only really a Camel implementation detail the distinguishing between the two of them. (In Camel you can configure on two levels - component or endpoint, hence why we have 2).

So in the salesforce-upsert-connector example you have both componentOptions and endpointOptions: https://github.com/redhat-ipaas/connectors/blob/master/connectors/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json#L15

There are 4 componentOptions and there are 2 endpointOptions which gives a total of 6 options. All these options are fined grained detailed in the other JSon file: https://github.com/redhat-ipaas/connectors/blob/master/connectors/salesforce-upsert-contact-connector/src/main/resources/camel-connector-schema.json

Now in this example and what is missing from the picture currently is that those 4 componentOptions are about login details and therefore should be marked as global options which we talk about in this ticket. So it may be that the iPaaS has this global configuration thingy, so what the citizen user will only be able to see/configure are the 2 other options.

In this timer connector example on the other hand there is only 1 option which is an endpointOption: https://github.com/redhat-ipaas/connectors/blob/master/connectors/timer-connector/src/main/resources/camel-connector.json - that option is named period.

That period option is further specified in the other json file, where you can find its type, default value, description, and so on: https://github.com/redhat-ipaas/connectors/blob/master/connectors/timer-connector/src/main/resources/camel-connector-schema.json

[davsclaus]

The connectors now have globalOptions defined

salesforce connector: https://github.com/redhat-ipaas/connectors/blob/master/connectors/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json#L15

twitter connector: https://github.com/redhat-ipaas/connectors/blob/master/connectors/twitter-mention-connector/src/main/resources/camel-connector.json#L15

All these options will be listed in the other JSon file as well where you can find the fine grained details. This is the same story for every option in the connector, they will be listed in that other JSon file.