search

apache / camel-kafka-connector

Camel Kafka Connector allows you to use all Camel components as Kafka Connect connectors
https://camel.apache.org
Apache License 2.0
154 stars 101 forks source link

Help Setting up Google Sheets Connector #1354

Closed jethomaspanw closed 2 months ago

jethomaspanw commented 2 years ago

Hi, I'm attempting to set up a Kafka Connector to consume from Google Sheets. I've been following the example found here: https://github.com/apache/camel-kafka-connector-examples/tree/main/google-sheets-stream/google-sheets-stream-source along with the documentation present here: https://camel.apache.org/camel-kafka-connector/1.0.x/reference/connectors/camel-google-sheets-source-kafka-source-connector.html

I am using camel-google-sheets-source-kafka-connector version 1.0.0, and deploying to a local minikube cluster.

I've been able to create the credentials with Google, but when deploying my connector locally with a Helm chart, I've run into several confusing errors.

When I leave camel.kamelet.google-sheets-source.range out of my Helm chart, I get an error claiming that range is a required parameter, even though the documentation lists it as optional.

When I edit my Helm chart to include a range value of A1:B5, the error message I get informs me that range is expected to be a boolean value, which makes no sense given the documented usage of the parameter, and doesn't match the example in the documentation.

When I edit my Helm chart to include range as a boolean set to true, the error message I get informs me that spreadsheetId is an unknown parameter, which is completely baffling, and has left me without a good direction to try next.

Does anyone here know how to create a camel-google-sheets-source-kafka-connector that can help me with setting the correct/expected parameters?

Helm Chart without Range: 

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: {{ .Values.name }}
  labels:
    # The strimzi.io/cluster label identifies the KafkaConnect instance
    # in which to create this connector. That KafkaConnect instance
    # must have the strimzi.io/use-connector-resources annotation
    # set to true.
    strimzi.io/cluster: {{ .Values.kafkaConnect.name }}
spec:
  class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
  tasksMax: {{ .Values.tasksMax }}
  config:
    topic: {{ .Values.topic }}
    camel.kamelet.google-sheets-source.accessToken: {{ .Values.accessToken }}
    camel.kamelet.google-sheets-source.applicationName: "Kafka_Connect_Google_Sheets_Source_Connector"
    camel.kamelet.google-sheets-source.clientId: {{ .Values.clientId }}
    camel.kamelet.google-sheets-source.clientSecret: {{ .Values.clientSecret }}
    camel.kamelet.google-sheets-source.index: 0
    camel.kamelet.google-sheets-source.refreshToken: {{ .Values.refreshToken }}
    camel.kamelet.google-sheets-source.spreadsheetId: {{ .Values.spreadsheetId }}

Result of kubectl describe kctr google-sheet-source-connector showing range is required: 

Name:         google-sheet-source-connector
Namespace:    my-kafka-project
Labels:       app.kubernetes.io/managed-by=Helm
              strimzi.io/cluster=my-connect-cluster
Annotations:  meta.helm.sh/release-name: google-sheets-connector
              meta.helm.sh/release-namespace: my-kafka-project
API Version:  kafka.strimzi.io/v1beta2
Kind:         KafkaConnector
Metadata:
  Creation Timestamp:  2022-04-04T22:14:45Z
  Generation:          1
  Managed Fields:
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:meta.helm.sh/release-name:
          f:meta.helm.sh/release-namespace:
        f:labels:
          .:
          f:app.kubernetes.io/managed-by:
          f:strimzi.io/cluster:
      f:spec:
        .:
        f:class:
        f:config:
          .:
          f:camel.kamelet.google-sheets-source.accessToken:
          f:camel.kamelet.google-sheets-source.applicationName:
          f:camel.kamelet.google-sheets-source.clientId:
          f:camel.kamelet.google-sheets-source.clientSecret:
          f:camel.kamelet.google-sheets-source.index:
          f:camel.kamelet.google-sheets-source.refreshToken:
          f:camel.kamelet.google-sheets-source.spreadsheetId:
          f:topic:
        f:tasksMax:
    Manager:      helm
    Operation:    Update
    Time:         2022-04-04T22:14:45Z
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:connectorStatus:
          .:
          f:connector:
            .:
            f:state:
            f:worker_id:
          f:name:
          f:tasks:
          f:type:
        f:observedGeneration:
        f:tasksMax:
        f:topics:
    Manager:         okhttp
    Operation:       Update
    Subresource:     status
    Time:            2022-04-04T22:14:46Z
  Resource Version:  23198
  UID:               8c218eb8-9796-4cf9-ac1c-899fcc070693
Spec:
  Class:  org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
  Config:
    camel.kamelet.google-sheets-source.accessToken:      XXXXXXXX
    camel.kamelet.google-sheets-source.applicationName:  Kafka_Connect_Google_Sheets_Source_Connector
    camel.kamelet.google-sheets-source.clientId:         XXXXXXXX
    camel.kamelet.google-sheets-source.clientSecret:     XXXXXXXX
    camel.kamelet.google-sheets-source.index:            0
    camel.kamelet.google-sheets-source.refreshToken:     XXXXXXXX
    camel.kamelet.google-sheets-source.spreadsheetId:    XXXXXXXX
    Topic:                                               google-sheet-topic
  Tasks Max:                                             2
Status:
  Conditions:
    Last Transition Time:  2022-04-04T22:15:30.662853Z
    Status:                True
    Type:                  Ready
  Connector Status:
    Connector:
      State:      RUNNING
      worker_id:  XXXXXXXX:XXX
    Name:         google-sheet-source-connector
    Tasks:
      Id:     0
      State:  FAILED
      Trace:  org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
              at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:185)
              at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
              at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
              at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
  at org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
  at org.apache.camel.support.service.BaseService.doFail(BaseService.java:413)
  at org.apache.camel.impl.engine.AbstractCamelContext.doFail(AbstractCamelContext.java:3482)
  at org.apache.camel.support.service.BaseService.fail(BaseService.java:342)
  at org.apache.camel.impl.engine.AbstractCamelContext.failOnStartup(AbstractCamelContext.java:5026)
  at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2579)
  at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
  at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2586)
  at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:247)
  at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
  at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
  at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:181)
  ... 8 more
Caused by: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:422)
  at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2870)
  at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
  at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2567)
  ... 14 more
Caused by: org.apache.camel.component.kamelet.KameletNotFoundException: Kamelet with id google-sheets-source not found in locations: classpath:/kamelets
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:410)
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:419)
  ... 17 more
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route google-sheets-source-33: Route(google-sheets-source-33)[From[google-sheets-stream://{... because of Failed to resolve endpoint: google-sheets-stream://{{index}}?accessToken=xxxxxx due to: Property with key [range] not found in properties from text: google-sheets-stream://0?accessToken=XXXXXXXX&applicationName=Kafka_Connect_Google_Sheets_Source_Connector&clientId=XXXXXXXX&clientSecret=XXXXXXXX&delay=500&range={{range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}&spreadsheetId={{spreadsheetId}}
  at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:80)
  at org.apache.camel.impl.DefaultModelReifierFactory.createRoute(DefaultModelReifierFactory.java:49)
  at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:868)
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:405)
  ... 18 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://{{index}}?accessToken=xxxxxx due to: Property with key [range] not found in properties from text: google-sheets-stream://0?accessToken=XXXXXXXX&applicationName=Kafka_Connect_Google_Sheets_Source_Connector&clientId=XXXXXXXX&clientSecret=XXXXXXXX&delay=500&range={{range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}&spreadsheetId={{spreadsheetId}}
  at org.apache.camel.support.EndpointHelper.resolveEndpointUriPropertyPlaceholders(EndpointHelper.java:86)
  at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:892)
  at org.apache.camel.impl.engine.AbstractCamelContext.getEndpoint(AbstractCamelContext.java:850)
  at org.apache.camel.support.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:58)
  at org.apache.camel.reifier.AbstractReifier.resolveEndpoint(AbstractReifier.java:199)
  at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:94)
  at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:74)
  ... 21 more
Caused by: java.lang.IllegalArgumentException: Property with key [range] not found in properties from text: google-sheets-stream://0?accessToken=XXXXXXXX&applicationName=Kafka_Connect_Google_Sheets_Source_Connector&clientId=XXXXXXXX&clientSecret=XXXXXXXX&delay=500&range={{range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}&spreadsheetId={{spreadsheetId}}
  at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.getPropertyValue(DefaultPropertiesParser.java:285)
  at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.readProperty(DefaultPropertiesParser.java:175)
  at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.doParse(DefaultPropertiesParser.java:117)
  at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.parse(DefaultPropertiesParser.java:101)
  at org.apache.camel.component.properties.DefaultPropertiesParser.parseUri(DefaultPropertiesParser.java:66)
  at org.apache.camel.component.properties.PropertiesComponent.parseUri(PropertiesComponent.java:274)
  at org.apache.camel.component.properties.PropertiesComponent.parseUri(PropertiesComponent.java:165)
  at org.apache.camel.impl.engine.AbstractCamelContext.resolvePropertyPlaceholders(AbstractCamelContext.java:1858)
  at org.apache.camel.support.EndpointHelper.resolveEndpointUriPropertyPlaceholders(EndpointHelper.java:74)
  ... 27 more

Helm Chart with Range A1:B5: 

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: {{ .Values.name }}
  labels:
    # The strimzi.io/cluster label identifies the KafkaConnect instance
    # in which to create this connector. That KafkaConnect instance
    # must have the strimzi.io/use-connector-resources annotation
    # set to true.
    strimzi.io/cluster: {{ .Values.kafkaConnect.name }}
spec:
  class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
  tasksMax: {{ .Values.tasksMax }}
  config:
    topic: {{ .Values.topic }}
    camel.kamelet.google-sheets-source.accessToken: {{ .Values.accessToken }}
    camel.kamelet.google-sheets-source.applicationName: "Kafka_Connect_Google_Sheets_Source_Connector"
    camel.kamelet.google-sheets-source.clientId: {{ .Values.clientId }}
    camel.kamelet.google-sheets-source.clientSecret: {{ .Values.clientSecret }}
    camel.kamelet.google-sheets-source.index: 0
    camel.kamelet.google-sheets-source.refreshToken: {{ .Values.refreshToken }}
    camel.kamelet.google-sheets-source.spreadsheetId: {{ .Values.spreadsheetId }}
    camel.kamelet.google-sheets-source.range: A1:B5

Result of kubectl describe kctr google-sheet-source-connector showing Range is expected to be a Boolean: 

Name:         google-sheet-source-connector
Namespace:    my-kafka-project
Labels:       app.kubernetes.io/managed-by=Helm
              strimzi.io/cluster=my-connect-cluster
Annotations:  meta.helm.sh/release-name: google-sheets-connector
              meta.helm.sh/release-namespace: my-kafka-project
API Version:  kafka.strimzi.io/v1beta2
Kind:         KafkaConnector
Metadata:
  Creation Timestamp:  2022-04-04T22:01:12Z
  Generation:          1
  Managed Fields:
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:meta.helm.sh/release-name:
          f:meta.helm.sh/release-namespace:
        f:labels:
          .:
          f:app.kubernetes.io/managed-by:
          f:strimzi.io/cluster:
      f:spec:
        .:
        f:class:
        f:config:
          .:
          f:camel.kamelet.google-sheets-source.accessToken:
          f:camel.kamelet.google-sheets-source.applicationName:
          f:camel.kamelet.google-sheets-source.clientId:
          f:camel.kamelet.google-sheets-source.clientSecret:
          f:camel.kamelet.google-sheets-source.index:
          f:camel.kamelet.google-sheets-source.range:
          f:camel.kamelet.google-sheets-source.refreshToken:
          f:camel.kamelet.google-sheets-source.spreadsheetId:
          f:topic:
        f:tasksMax:
    Manager:      helm
    Operation:    Update
    Time:         2022-04-04T22:01:12Z
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:observedGeneration:
        f:tasksMax:
        f:topics:
    Manager:         okhttp
    Operation:       Update
    Subresource:     status
    Time:            2022-04-04T22:01:13Z
  Resource Version:  22573
  UID:               267d436e-cad7-4226-bb61-7e1bdb473127
Spec:
  Class:  org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
  Config:
    camel.kamelet.google-sheets-source.accessToken:     XXXXXXXX
    camel.kamelet.google-sheets-source.applicationName:  Kafka_Connect_Google_Sheets_Source_Connector
    camel.kamelet.google-sheets-source.clientId:         XXXXXXXXX
    camel.kamelet.google-sheets-source.clientSecret:     XXXXXXXX
    camel.kamelet.google-sheets-source.index:            0
    camel.kamelet.google-sheets-source.range:            A1:B5
    camel.kamelet.google-sheets-source.refreshToken:     XXXXXXX
    camel.kamelet.google-sheets-source.spreadsheetId:   XXXXXXXXX
    Topic:                                               google-sheet-topic
  Tasks Max:                                             2
Status:
  Conditions:
    Last Transition Time:  2022-04-04T22:01:13.102851Z
    Message:               PUT /connectors/google-sheet-source-connector/config returned 400 (Bad Request): Connector configuration is invalid and contains the following 1 error(s):
Invalid value A1:B5 for configuration camel.kamelet.google-sheets-source.range: Expected value to be either true or false
You can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`
    Reason:             ConnectRestException
    Status:             True
    Type:               NotReady
  Observed Generation:  1
  Tasks Max:            2
  Topics:
Events:  <none>

Helm Chart with Range as Boolean:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: {{ .Values.name }}
  labels:
    # The strimzi.io/cluster label identifies the KafkaConnect instance
    # in which to create this connector. That KafkaConnect instance
    # must have the strimzi.io/use-connector-resources annotation
    # set to true.
    strimzi.io/cluster: {{ .Values.kafkaConnect.name }}
spec:
  class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
  tasksMax: {{ .Values.tasksMax }}
  config:
    topic: {{ .Values.topic }}
    camel.kamelet.google-sheets-source.accessToken: {{ .Values.accessToken }}
    camel.kamelet.google-sheets-source.applicationName: "Kafka_Connect_Google_Sheets_Source_Connector"
    camel.kamelet.google-sheets-source.clientId: {{ .Values.clientId }}
    camel.kamelet.google-sheets-source.clientSecret: {{ .Values.clientSecret }}
    camel.kamelet.google-sheets-source.index: 0
    camel.kamelet.google-sheets-source.refreshToken: {{ .Values.refreshToken }}
    camel.kamelet.google-sheets-source.spreadsheetId: {{ .Values.spreadsheetId }}
    camel.kamelet.google-sheets-source.range: true

Result of kubectl describe kctr google-sheet-source-connector showing spreadsheetId is not a recognized parameter:

Name:         google-sheet-source-connector
Namespace:    my-kafka-project
Labels:       app.kubernetes.io/managed-by=Helm
              strimzi.io/cluster=my-connect-cluster
Annotations:  meta.helm.sh/release-name: google-sheets-connector
              meta.helm.sh/release-namespace: my-kafka-project
API Version:  kafka.strimzi.io/v1beta2
Kind:         KafkaConnector
Metadata:
  Creation Timestamp:  2022-04-04T22:23:15Z
  Generation:          1
  Managed Fields:
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:meta.helm.sh/release-name:
          f:meta.helm.sh/release-namespace:
        f:labels:
          .:
          f:app.kubernetes.io/managed-by:
          f:strimzi.io/cluster:
      f:spec:
        .:
        f:class:
        f:config:
          .:
          f:camel.kamelet.google-sheets-source.accessToken:
          f:camel.kamelet.google-sheets-source.applicationName:
          f:camel.kamelet.google-sheets-source.clientId:
          f:camel.kamelet.google-sheets-source.clientSecret:
          f:camel.kamelet.google-sheets-source.index:
          f:camel.kamelet.google-sheets-source.range:
          f:camel.kamelet.google-sheets-source.refreshToken:
          f:camel.kamelet.google-sheets-source.spreadsheetId:
          f:topic:
        f:tasksMax:
    Manager:      helm
    Operation:    Update
    Time:         2022-04-04T22:23:15Z
    API Version:  kafka.strimzi.io/v1beta2
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:connectorStatus:
          .:
          f:connector:
            .:
            f:state:
            f:worker_id:
          f:name:
          f:tasks:
          f:type:
        f:observedGeneration:
        f:tasksMax:
        f:topics:
    Manager:         okhttp
    Operation:       Update
    Subresource:     status
    Time:            2022-04-04T22:23:15Z
  Resource Version:  23531
  UID:               73812ecf-0187-41f0-8245-e4d5f2f72f10
Spec:
  Class:  org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
  Config:
    camel.kamelet.google-sheets-source.accessToken:      XXXXXXXX
    camel.kamelet.google-sheets-source.applicationName:  Kafka_Connect_Google_Sheets_Source_Connector
    camel.kamelet.google-sheets-source.clientId:         XXXXXXXX
    camel.kamelet.google-sheets-source.clientSecret:     XXXXXXXX
    camel.kamelet.google-sheets-source.index:            0
    camel.kamelet.google-sheets-source.range:            true
    camel.kamelet.google-sheets-source.refreshToken:     XXXXXXXX
    camel.kamelet.google-sheets-source.spreadsheetId:    XXXXXXXX
    Topic:                                               google-sheet-topic
  Tasks Max:                                             2
Status:
  Conditions:
    Last Transition Time:  2022-04-04T22:23:15.662456Z
    Status:                True
    Type:                  Ready
  Connector Status:
    Connector:
      State:      RUNNING
      worker_id:  XXXXXXXX:XXX
    Name:         google-sheet-source-connector
    Tasks:
      Id:     0
      State:  FAILED
      Trace:  org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
              at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:185)
              at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
              at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
              at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
  at org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
  at org.apache.camel.support.service.BaseService.doFail(BaseService.java:413)
  at org.apache.camel.impl.engine.AbstractCamelContext.doFail(AbstractCamelContext.java:3482)
  at org.apache.camel.support.service.BaseService.fail(BaseService.java:342)
  at org.apache.camel.impl.engine.AbstractCamelContext.failOnStartup(AbstractCamelContext.java:5026)
  at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2579)
  at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
  at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2586)
  at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:247)
  at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
  at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
  at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:181)
  ... 8 more
Caused by: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:422)
  at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2870)
  at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
  at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2567)
  ... 14 more
Caused by: org.apache.camel.component.kamelet.KameletNotFoundException: Kamelet with id google-sheets-source not found in locations: classpath:/kamelets
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:410)
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:419)
  ... 17 more
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route google-sheets-source-39: Route(google-sheets-source-39)[From[google-sheets-stream://{... because of Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
  at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:80)
  at org.apache.camel.impl.DefaultModelReifierFactory.createRoute(DefaultModelReifierFactory.java:49)
  at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:868)
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:405)
  ... 18 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
  at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:968)
  at org.apache.camel.impl.engine.AbstractCamelContext.getEndpoint(AbstractCamelContext.java:850)
  at org.apache.camel.support.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:58)
  at org.apache.camel.reifier.AbstractReifier.resolveEndpoint(AbstractReifier.java:199)
  at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:94)
  at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:74)
  ... 21 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
  at org.apache.camel.support.DefaultComponent.validateParameters(DefaultComponent.java:299)
  at org.apache.camel.support.DefaultComponent.createEndpoint(DefaultComponent.java:192)
  at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:934)
  ... 26 more

      worker_id:  XXXXXXXX:XXX
      Id:         1
      State:      FAILED
      Trace:      org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
                  at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:185)
                  at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
                  at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
                  at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
                  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
                  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
                  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
                  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
                  at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
  at org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
  at org.apache.camel.support.service.BaseService.doFail(BaseService.java:413)
  at org.apache.camel.impl.engine.AbstractCamelContext.doFail(AbstractCamelContext.java:3482)
  at org.apache.camel.support.service.BaseService.fail(BaseService.java:342)
  at org.apache.camel.impl.engine.AbstractCamelContext.failOnStartup(AbstractCamelContext.java:5026)
  at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2579)
  at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
  at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2586)
  at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:247)
  at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
  at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
  at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:181)
  ... 8 more
Caused by: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:422)
  at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2870)
  at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
  at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2567)
  ... 14 more
Caused by: org.apache.camel.component.kamelet.KameletNotFoundException: Kamelet with id google-sheets-source not found in locations: classpath:/kamelets
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:410)
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:419)
  ... 17 more
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route google-sheets-source-37: Route(google-sheets-source-37)[From[google-sheets-stream://{... because of Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
  at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:80)
  at org.apache.camel.impl.DefaultModelReifierFactory.createRoute(DefaultModelReifierFactory.java:49)
  at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:868)
  at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:405)
  ... 18 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
  at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:968)
  at org.apache.camel.impl.engine.AbstractCamelContext.getEndpoint(AbstractCamelContext.java:850)
  at org.apache.camel.support.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:58)
  at org.apache.camel.reifier.AbstractReifier.resolveEndpoint(AbstractReifier.java:199)
  at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:94)
  at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:74)
  ... 21 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
  at org.apache.camel.support.DefaultComponent.validateParameters(DefaultComponent.java:299)
  at org.apache.camel.support.DefaultComponent.createEndpoint(DefaultComponent.java:192)
  at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:934)
  ... 26 more

      worker_id:        XXXXXXXX:XXX
    Type:               source
  Observed Generation:  1
  Tasks Max:            2
  Topics:
Events:  <none>
oscerd commented 2 years ago

The range type seems to be wrong in the related kamelet: https://github.com/apache/camel-kamelets/blob/main/kamelets/google-sheets-source.kamelet.yaml#L107

oscerd commented 2 years ago

I need to test a bit.

oscerd commented 2 months ago

This should be fine now with 4.4.2