Use Camel upstream

[christophd]

This is a...

[x] Feature request
[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report  
[ ] Documentation issue or request

Description

Syndesis community version should use the Camel upstream version. Camel 2.24.2 should be the latest version (Need to check if all Camel components used in Syndesis are available in this version).

Coming from https://github.com/syndesisio/syndesis/wiki/Meeting-Notes

[heiko-braun]

@christophd Please talk to @oscerd about this. I think there was good reason not to align to the most recent 2.x in the past.

[Delawen]

Comparing current Syndesis master connectors (folder app/connector) with Camel 2.24.x (folder components):

activemq -> camel-activemq amqp -> camel-amqp api-provider -> ??? no need for camel connector? aws-ddb -> camel-aws-ddb aws-s3 -> camel-aws-s3 aws-sns -> camel-aws-sns aws-sqs -> camel-aws-sqs box -> camel-box concur -> ???? no need for camel connector? dropbox -> camel-dropbox email -> camel-mail fhir -> camel-fhir flow -> ??? no need for camel connector? ftp -> camel-ftp gmail -> camel-google-mail google-calendar -> camel-google-calendar google-sheets -> camel-google-sheets http -> camel-http irc -> camel-irc jira -> camel-jira kafka -> camel-kafkaq knative -> ??? no need for specific camel connector? kudu -> ??? log -> camel-log mongodb -> camel-mongodb mqtt -> camel-mqtt odata -> camel-olingo4 rest-swagger -> camel-rest-swagger salesforce -> camel-salesforce servicenow -> camel-servicenow sftp -> camel-ftp (why then a different Syndesis connector?) slack -> camel-slack sql -> camel-sql telegram -> camel-telegram timer -> camel-timer twitter -> camel-twitter webhook -> camel-webhook

This is a quick comparison just by name, but it looks that there are several components we should port to Camel.

[Delawen]

Do we keep track of any improvement/change on components of Camel-Fuse that should be ported too?

[Delawen]

I can start working on porting anything missing, but I need a list of things to port. If there's no list available, I can start checking manually but that's going to take time.

[zregvart]

Syndesis connector is not a Camel component, so comparing the list of connectors with Camel components will not show much.

This command:

find . -path '*/src/main/resources/META-INF/syndesis/connector/*.json' -exec jq .actions[].descriptor.componentScheme {} \;|sort -u

Will give the list of component schemes used, which usually corresponds to the camel component name.

Out of those only Camel Kudu component is hosted within Syndesis and within org.apache.camel package which we should not do. This was done in the interest of time at the moment.

[Delawen]

Is there any good reason not to migrate Kudu to Camel?

(that command was much better than my sad grep :D)

[zregvart]

There was a valid reason to do it when we were developing the Kudu connector, I don't think there is one now. Or for that matter that we should question if a component should be contributed to Camel.

[oscerd]

You can't backport new components on 2.24.x

[oscerd]

You can eventually try on camel-2.x for 2.25.0

[oscerd]

There is this issue for contribute upstream the kudu component https://github.com/syndesisio/syndesis/issues/4457

[oscerd]

My suggestion is to first open an upstream issue, then create a PR for master, then creating a PR for 2.x. Adding a component is not really straightforward, you need to think about multiple aspects.

[Delawen]

Is there an interest in Apache Camel community to have a version 2.25.0 or all efforts are moving towards version 3?

Because maybe it makes more sense to see how far we are from using version 3.

[oscerd]

trust me, Syndesis is really far from using version 3. By the way 2.25.0 is in the roadmap, so we should release it soon.

[Delawen]

Then it's settled :)

[squakez]

If it helps, this is the list of failing connectors when setting 2.24.0

[INFO] Connector :: AWS SQS ............................... FAILURE [  2.478 s]
[INFO] Connector :: Gmail ................................. FAILURE [  3.202 s]
[INFO] Connector :: Google Sheets ......................... FAILURE [  3.331 s]
[INFO] Connector :: Jira .................................. FAILURE [  4.026 s]

Also Connector-Kafka is failing because a driver dependency is missing in its pom (likely it inherit the camel upstream version one).

As far I can see, there is no need to immediately create a kudu component to upstream, as it seems that the connector kudu development haven't used any. If you create the upstream component you'll have to rewrite the connector-kudu as well.

[squakez]

If it helps, here the list of failures that we should fix to port to 2.24.0, at least for connector compilation: log.txt

[Delawen]

Very interesting :)

[christophd]

@claudio4j might know regarding JIRA connector

[oscerd]

The camel-jira in camel-2.x is the old one. The component from @claudio4j is a completely new one, available only on camel 3. We cannot break existing users on the old one, so backporting the new one is not feasible at all, at least, not with the camel-jira name.

[Delawen]

So we are going to focus on 2.25 and give back to camel upstream everything that we can from fuse. And what can't be migrated, can live on Syndesis source code.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[Delawen]

Not stalled! Just long :)

[Delawen]

O guess we can consider this finished? (Besides the pending specific issues on master).

[zregvart]

Yeah, let's close this one and create new ones for specific issues.