I am using camel-google-sheets-source-kafka-connector version 1.0.0, and deploying to a local minikube cluster.
I've been able to create the credentials with Google, but when deploying my connector locally with a Helm chart, I've run into several confusing errors.
When I leave camel.kamelet.google-sheets-source.range out of my Helm chart, I get an error claiming that range is a required parameter, even though the documentation lists it as optional.
When I edit my Helm chart to include a range value of A1:B5, the error message I get informs me that range is expected to be a boolean value, which makes no sense given the documented usage of the parameter, and doesn't match the example in the documentation.
When I edit my Helm chart to include range as a boolean set to true, the error message I get informs me that spreadsheetId is an unknown parameter, which is completely baffling, and has left me without a good direction to try next.
Does anyone here know how to create a camel-google-sheets-source-kafka-connector that can help me with setting the correct/expected parameters?
Helm Chart without Range:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: {{ .Values.name }}
labels:
# The strimzi.io/cluster label identifies the KafkaConnect instance
# in which to create this connector. That KafkaConnect instance
# must have the strimzi.io/use-connector-resources annotation
# set to true.
strimzi.io/cluster: {{ .Values.kafkaConnect.name }}
spec:
class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
tasksMax: {{ .Values.tasksMax }}
config:
topic: {{ .Values.topic }}
camel.kamelet.google-sheets-source.accessToken: {{ .Values.accessToken }}
camel.kamelet.google-sheets-source.applicationName: "Kafka_Connect_Google_Sheets_Source_Connector"
camel.kamelet.google-sheets-source.clientId: {{ .Values.clientId }}
camel.kamelet.google-sheets-source.clientSecret: {{ .Values.clientSecret }}
camel.kamelet.google-sheets-source.index: 0
camel.kamelet.google-sheets-source.refreshToken: {{ .Values.refreshToken }}
camel.kamelet.google-sheets-source.spreadsheetId: {{ .Values.spreadsheetId }}
Result of kubectl describe kctr google-sheet-source-connector showing range is required:
Name: google-sheet-source-connector
Namespace: my-kafka-project
Labels: app.kubernetes.io/managed-by=Helm
strimzi.io/cluster=my-connect-cluster
Annotations: meta.helm.sh/release-name: google-sheets-connector
meta.helm.sh/release-namespace: my-kafka-project
API Version: kafka.strimzi.io/v1beta2
Kind: KafkaConnector
Metadata:
Creation Timestamp: 2022-04-04T22:14:45Z
Generation: 1
Managed Fields:
API Version: kafka.strimzi.io/v1beta2
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:meta.helm.sh/release-name:
f:meta.helm.sh/release-namespace:
f:labels:
.:
f:app.kubernetes.io/managed-by:
f:strimzi.io/cluster:
f:spec:
.:
f:class:
f:config:
.:
f:camel.kamelet.google-sheets-source.accessToken:
f:camel.kamelet.google-sheets-source.applicationName:
f:camel.kamelet.google-sheets-source.clientId:
f:camel.kamelet.google-sheets-source.clientSecret:
f:camel.kamelet.google-sheets-source.index:
f:camel.kamelet.google-sheets-source.refreshToken:
f:camel.kamelet.google-sheets-source.spreadsheetId:
f:topic:
f:tasksMax:
Manager: helm
Operation: Update
Time: 2022-04-04T22:14:45Z
API Version: kafka.strimzi.io/v1beta2
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:conditions:
f:connectorStatus:
.:
f:connector:
.:
f:state:
f:worker_id:
f:name:
f:tasks:
f:type:
f:observedGeneration:
f:tasksMax:
f:topics:
Manager: okhttp
Operation: Update
Subresource: status
Time: 2022-04-04T22:14:46Z
Resource Version: 23198
UID: 8c218eb8-9796-4cf9-ac1c-899fcc070693
Spec:
Class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
Config:
camel.kamelet.google-sheets-source.accessToken: XXXXXXXX
camel.kamelet.google-sheets-source.applicationName: Kafka_Connect_Google_Sheets_Source_Connector
camel.kamelet.google-sheets-source.clientId: XXXXXXXX
camel.kamelet.google-sheets-source.clientSecret: XXXXXXXX
camel.kamelet.google-sheets-source.index: 0
camel.kamelet.google-sheets-source.refreshToken: XXXXXXXX
camel.kamelet.google-sheets-source.spreadsheetId: XXXXXXXX
Topic: google-sheet-topic
Tasks Max: 2
Status:
Conditions:
Last Transition Time: 2022-04-04T22:15:30.662853Z
Status: True
Type: Ready
Connector Status:
Connector:
State: RUNNING
worker_id: XXXXXXXX:XXX
Name: google-sheet-source-connector
Tasks:
Id: 0
State: FAILED
Trace: org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:185)
at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
at org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
at org.apache.camel.support.service.BaseService.doFail(BaseService.java:413)
at org.apache.camel.impl.engine.AbstractCamelContext.doFail(AbstractCamelContext.java:3482)
at org.apache.camel.support.service.BaseService.fail(BaseService.java:342)
at org.apache.camel.impl.engine.AbstractCamelContext.failOnStartup(AbstractCamelContext.java:5026)
at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2579)
at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2586)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:247)
at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:181)
... 8 more
Caused by: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:422)
at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2870)
at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2567)
... 14 more
Caused by: org.apache.camel.component.kamelet.KameletNotFoundException: Kamelet with id google-sheets-source not found in locations: classpath:/kamelets
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:410)
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:419)
... 17 more
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route google-sheets-source-33: Route(google-sheets-source-33)[From[google-sheets-stream://{... because of Failed to resolve endpoint: google-sheets-stream://{{index}}?accessToken=xxxxxx due to: Property with key [range] not found in properties from text: google-sheets-stream://0?accessToken=XXXXXXXX&applicationName=Kafka_Connect_Google_Sheets_Source_Connector&clientId=XXXXXXXX&clientSecret=XXXXXXXX&delay=500&range={{range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}&spreadsheetId={{spreadsheetId}}
at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:80)
at org.apache.camel.impl.DefaultModelReifierFactory.createRoute(DefaultModelReifierFactory.java:49)
at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:868)
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:405)
... 18 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://{{index}}?accessToken=xxxxxx due to: Property with key [range] not found in properties from text: google-sheets-stream://0?accessToken=XXXXXXXX&applicationName=Kafka_Connect_Google_Sheets_Source_Connector&clientId=XXXXXXXX&clientSecret=XXXXXXXX&delay=500&range={{range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}&spreadsheetId={{spreadsheetId}}
at org.apache.camel.support.EndpointHelper.resolveEndpointUriPropertyPlaceholders(EndpointHelper.java:86)
at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:892)
at org.apache.camel.impl.engine.AbstractCamelContext.getEndpoint(AbstractCamelContext.java:850)
at org.apache.camel.support.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:58)
at org.apache.camel.reifier.AbstractReifier.resolveEndpoint(AbstractReifier.java:199)
at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:94)
at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:74)
... 21 more
Caused by: java.lang.IllegalArgumentException: Property with key [range] not found in properties from text: google-sheets-stream://0?accessToken=XXXXXXXX&applicationName=Kafka_Connect_Google_Sheets_Source_Connector&clientId=XXXXXXXX&clientSecret=XXXXXXXX&delay=500&range={{range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}&spreadsheetId={{spreadsheetId}}
at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.getPropertyValue(DefaultPropertiesParser.java:285)
at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.readProperty(DefaultPropertiesParser.java:175)
at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.doParse(DefaultPropertiesParser.java:117)
at org.apache.camel.component.properties.DefaultPropertiesParser$ParsingContext.parse(DefaultPropertiesParser.java:101)
at org.apache.camel.component.properties.DefaultPropertiesParser.parseUri(DefaultPropertiesParser.java:66)
at org.apache.camel.component.properties.PropertiesComponent.parseUri(PropertiesComponent.java:274)
at org.apache.camel.component.properties.PropertiesComponent.parseUri(PropertiesComponent.java:165)
at org.apache.camel.impl.engine.AbstractCamelContext.resolvePropertyPlaceholders(AbstractCamelContext.java:1858)
at org.apache.camel.support.EndpointHelper.resolveEndpointUriPropertyPlaceholders(EndpointHelper.java:74)
... 27 more
Helm Chart with Range A1:B5:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: {{ .Values.name }}
labels:
# The strimzi.io/cluster label identifies the KafkaConnect instance
# in which to create this connector. That KafkaConnect instance
# must have the strimzi.io/use-connector-resources annotation
# set to true.
strimzi.io/cluster: {{ .Values.kafkaConnect.name }}
spec:
class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
tasksMax: {{ .Values.tasksMax }}
config:
topic: {{ .Values.topic }}
camel.kamelet.google-sheets-source.accessToken: {{ .Values.accessToken }}
camel.kamelet.google-sheets-source.applicationName: "Kafka_Connect_Google_Sheets_Source_Connector"
camel.kamelet.google-sheets-source.clientId: {{ .Values.clientId }}
camel.kamelet.google-sheets-source.clientSecret: {{ .Values.clientSecret }}
camel.kamelet.google-sheets-source.index: 0
camel.kamelet.google-sheets-source.refreshToken: {{ .Values.refreshToken }}
camel.kamelet.google-sheets-source.spreadsheetId: {{ .Values.spreadsheetId }}
camel.kamelet.google-sheets-source.range: A1:B5
Result of kubectl describe kctr google-sheet-source-connector showing Range is expected to be a Boolean:
Name: google-sheet-source-connector
Namespace: my-kafka-project
Labels: app.kubernetes.io/managed-by=Helm
strimzi.io/cluster=my-connect-cluster
Annotations: meta.helm.sh/release-name: google-sheets-connector
meta.helm.sh/release-namespace: my-kafka-project
API Version: kafka.strimzi.io/v1beta2
Kind: KafkaConnector
Metadata:
Creation Timestamp: 2022-04-04T22:01:12Z
Generation: 1
Managed Fields:
API Version: kafka.strimzi.io/v1beta2
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:meta.helm.sh/release-name:
f:meta.helm.sh/release-namespace:
f:labels:
.:
f:app.kubernetes.io/managed-by:
f:strimzi.io/cluster:
f:spec:
.:
f:class:
f:config:
.:
f:camel.kamelet.google-sheets-source.accessToken:
f:camel.kamelet.google-sheets-source.applicationName:
f:camel.kamelet.google-sheets-source.clientId:
f:camel.kamelet.google-sheets-source.clientSecret:
f:camel.kamelet.google-sheets-source.index:
f:camel.kamelet.google-sheets-source.range:
f:camel.kamelet.google-sheets-source.refreshToken:
f:camel.kamelet.google-sheets-source.spreadsheetId:
f:topic:
f:tasksMax:
Manager: helm
Operation: Update
Time: 2022-04-04T22:01:12Z
API Version: kafka.strimzi.io/v1beta2
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:conditions:
f:observedGeneration:
f:tasksMax:
f:topics:
Manager: okhttp
Operation: Update
Subresource: status
Time: 2022-04-04T22:01:13Z
Resource Version: 22573
UID: 267d436e-cad7-4226-bb61-7e1bdb473127
Spec:
Class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
Config:
camel.kamelet.google-sheets-source.accessToken: XXXXXXXX
camel.kamelet.google-sheets-source.applicationName: Kafka_Connect_Google_Sheets_Source_Connector
camel.kamelet.google-sheets-source.clientId: XXXXXXXXX
camel.kamelet.google-sheets-source.clientSecret: XXXXXXXX
camel.kamelet.google-sheets-source.index: 0
camel.kamelet.google-sheets-source.range: A1:B5
camel.kamelet.google-sheets-source.refreshToken: XXXXXXX
camel.kamelet.google-sheets-source.spreadsheetId: XXXXXXXXX
Topic: google-sheet-topic
Tasks Max: 2
Status:
Conditions:
Last Transition Time: 2022-04-04T22:01:13.102851Z
Message: PUT /connectors/google-sheet-source-connector/config returned 400 (Bad Request): Connector configuration is invalid and contains the following 1 error(s):
Invalid value A1:B5 for configuration camel.kamelet.google-sheets-source.range: Expected value to be either true or false
You can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`
Reason: ConnectRestException
Status: True
Type: NotReady
Observed Generation: 1
Tasks Max: 2
Topics:
Events: <none>
Helm Chart with Range as Boolean:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: {{ .Values.name }}
labels:
# The strimzi.io/cluster label identifies the KafkaConnect instance
# in which to create this connector. That KafkaConnect instance
# must have the strimzi.io/use-connector-resources annotation
# set to true.
strimzi.io/cluster: {{ .Values.kafkaConnect.name }}
spec:
class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
tasksMax: {{ .Values.tasksMax }}
config:
topic: {{ .Values.topic }}
camel.kamelet.google-sheets-source.accessToken: {{ .Values.accessToken }}
camel.kamelet.google-sheets-source.applicationName: "Kafka_Connect_Google_Sheets_Source_Connector"
camel.kamelet.google-sheets-source.clientId: {{ .Values.clientId }}
camel.kamelet.google-sheets-source.clientSecret: {{ .Values.clientSecret }}
camel.kamelet.google-sheets-source.index: 0
camel.kamelet.google-sheets-source.refreshToken: {{ .Values.refreshToken }}
camel.kamelet.google-sheets-source.spreadsheetId: {{ .Values.spreadsheetId }}
camel.kamelet.google-sheets-source.range: true
Result of kubectl describe kctr google-sheet-source-connector showing spreadsheetId is not a recognized parameter:
Name: google-sheet-source-connector
Namespace: my-kafka-project
Labels: app.kubernetes.io/managed-by=Helm
strimzi.io/cluster=my-connect-cluster
Annotations: meta.helm.sh/release-name: google-sheets-connector
meta.helm.sh/release-namespace: my-kafka-project
API Version: kafka.strimzi.io/v1beta2
Kind: KafkaConnector
Metadata:
Creation Timestamp: 2022-04-04T22:23:15Z
Generation: 1
Managed Fields:
API Version: kafka.strimzi.io/v1beta2
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:meta.helm.sh/release-name:
f:meta.helm.sh/release-namespace:
f:labels:
.:
f:app.kubernetes.io/managed-by:
f:strimzi.io/cluster:
f:spec:
.:
f:class:
f:config:
.:
f:camel.kamelet.google-sheets-source.accessToken:
f:camel.kamelet.google-sheets-source.applicationName:
f:camel.kamelet.google-sheets-source.clientId:
f:camel.kamelet.google-sheets-source.clientSecret:
f:camel.kamelet.google-sheets-source.index:
f:camel.kamelet.google-sheets-source.range:
f:camel.kamelet.google-sheets-source.refreshToken:
f:camel.kamelet.google-sheets-source.spreadsheetId:
f:topic:
f:tasksMax:
Manager: helm
Operation: Update
Time: 2022-04-04T22:23:15Z
API Version: kafka.strimzi.io/v1beta2
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:conditions:
f:connectorStatus:
.:
f:connector:
.:
f:state:
f:worker_id:
f:name:
f:tasks:
f:type:
f:observedGeneration:
f:tasksMax:
f:topics:
Manager: okhttp
Operation: Update
Subresource: status
Time: 2022-04-04T22:23:15Z
Resource Version: 23531
UID: 73812ecf-0187-41f0-8245-e4d5f2f72f10
Spec:
Class: org.apache.camel.kafkaconnector.googlesheetssource.CamelGooglesheetssourceSourceConnector
Config:
camel.kamelet.google-sheets-source.accessToken: XXXXXXXX
camel.kamelet.google-sheets-source.applicationName: Kafka_Connect_Google_Sheets_Source_Connector
camel.kamelet.google-sheets-source.clientId: XXXXXXXX
camel.kamelet.google-sheets-source.clientSecret: XXXXXXXX
camel.kamelet.google-sheets-source.index: 0
camel.kamelet.google-sheets-source.range: true
camel.kamelet.google-sheets-source.refreshToken: XXXXXXXX
camel.kamelet.google-sheets-source.spreadsheetId: XXXXXXXX
Topic: google-sheet-topic
Tasks Max: 2
Status:
Conditions:
Last Transition Time: 2022-04-04T22:23:15.662456Z
Status: True
Type: Ready
Connector Status:
Connector:
State: RUNNING
worker_id: XXXXXXXX:XXX
Name: google-sheet-source-connector
Tasks:
Id: 0
State: FAILED
Trace: org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:185)
at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
at org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
at org.apache.camel.support.service.BaseService.doFail(BaseService.java:413)
at org.apache.camel.impl.engine.AbstractCamelContext.doFail(AbstractCamelContext.java:3482)
at org.apache.camel.support.service.BaseService.fail(BaseService.java:342)
at org.apache.camel.impl.engine.AbstractCamelContext.failOnStartup(AbstractCamelContext.java:5026)
at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2579)
at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2586)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:247)
at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:181)
... 8 more
Caused by: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:422)
at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2870)
at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2567)
... 14 more
Caused by: org.apache.camel.component.kamelet.KameletNotFoundException: Kamelet with id google-sheets-source not found in locations: classpath:/kamelets
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:410)
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:419)
... 17 more
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route google-sheets-source-39: Route(google-sheets-source-39)[From[google-sheets-stream://{... because of Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:80)
at org.apache.camel.impl.DefaultModelReifierFactory.createRoute(DefaultModelReifierFactory.java:49)
at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:868)
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:405)
... 18 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:968)
at org.apache.camel.impl.engine.AbstractCamelContext.getEndpoint(AbstractCamelContext.java:850)
at org.apache.camel.support.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:58)
at org.apache.camel.reifier.AbstractReifier.resolveEndpoint(AbstractReifier.java:199)
at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:94)
at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:74)
... 21 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
at org.apache.camel.support.DefaultComponent.validateParameters(DefaultComponent.java:299)
at org.apache.camel.support.DefaultComponent.createEndpoint(DefaultComponent.java:192)
at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:934)
... 26 more
worker_id: XXXXXXXX:XXX
Id: 1
State: FAILED
Trace: org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:185)
at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
at org.apache.camel.RuntimeCamelException.wrapRuntimeException(RuntimeCamelException.java:66)
at org.apache.camel.support.service.BaseService.doFail(BaseService.java:413)
at org.apache.camel.impl.engine.AbstractCamelContext.doFail(AbstractCamelContext.java:3482)
at org.apache.camel.support.service.BaseService.fail(BaseService.java:342)
at org.apache.camel.impl.engine.AbstractCamelContext.failOnStartup(AbstractCamelContext.java:5026)
at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2579)
at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2586)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:247)
at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:181)
... 8 more
Caused by: org.apache.camel.VetoCamelContextStartException: Failure creating route from template: google-sheets-source
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:422)
at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2870)
at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2567)
... 14 more
Caused by: org.apache.camel.component.kamelet.KameletNotFoundException: Kamelet with id google-sheets-source not found in locations: classpath:/kamelets
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:410)
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.onContextInitialized(KameletComponent.java:419)
... 17 more
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route google-sheets-source-37: Route(google-sheets-source-37)[From[google-sheets-stream://{... because of Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:80)
at org.apache.camel.impl.DefaultModelReifierFactory.createRoute(DefaultModelReifierFactory.java:49)
at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:868)
at org.apache.camel.component.kamelet.KameletComponent$LifecycleHandler.createRouteForEndpoint(KameletComponent.java:405)
... 18 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:968)
at org.apache.camel.impl.engine.AbstractCamelContext.getEndpoint(AbstractCamelContext.java:850)
at org.apache.camel.support.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:58)
at org.apache.camel.reifier.AbstractReifier.resolveEndpoint(AbstractReifier.java:199)
at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:94)
at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:74)
... 21 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: google-sheets-stream://0?accessToken=xxxxxx&splitResults=true&spreadsheetId=XXXXXXXXXX due to: There are 1 parameters that couldn't be set on the endpoint. Check the uri if the parameters are spelt correctly and that they are properties of the endpoint. Unknown parameters=[{spreadsheetId=XXXXXXXXXX}]
at org.apache.camel.support.DefaultComponent.validateParameters(DefaultComponent.java:299)
at org.apache.camel.support.DefaultComponent.createEndpoint(DefaultComponent.java:192)
at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:934)
... 26 more
worker_id: XXXXXXXX:XXX
Type: source
Observed Generation: 1
Tasks Max: 2
Topics:
Events: <none>
Hi, I'm attempting to set up a Kafka Connector to consume from Google Sheets. I've been following the example found here: https://github.com/apache/camel-kafka-connector-examples/tree/main/google-sheets-stream/google-sheets-stream-source along with the documentation present here: https://camel.apache.org/camel-kafka-connector/1.0.x/reference/connectors/camel-google-sheets-source-kafka-source-connector.html
I am using camel-google-sheets-source-kafka-connector version 1.0.0, and deploying to a local minikube cluster.
I've been able to create the credentials with Google, but when deploying my connector locally with a Helm chart, I've run into several confusing errors.
When I leave camel.kamelet.google-sheets-source.range out of my Helm chart, I get an error claiming that range is a required parameter, even though the documentation lists it as optional.
When I edit my Helm chart to include a range value of A1:B5, the error message I get informs me that range is expected to be a boolean value, which makes no sense given the documented usage of the parameter, and doesn't match the example in the documentation.
When I edit my Helm chart to include range as a boolean set to true, the error message I get informs me that spreadsheetId is an unknown parameter, which is completely baffling, and has left me without a good direction to try next.
Does anyone here know how to create a camel-google-sheets-source-kafka-connector that can help me with setting the correct/expected parameters?
Helm Chart without Range:
Result of
kubectl describe kctr google-sheet-source-connector
showing range is required:Helm Chart with Range A1:B5:
Result of
kubectl describe kctr google-sheet-source-connector
showing Range is expected to be a Boolean:Helm Chart with Range as Boolean:
Result of
kubectl describe kctr google-sheet-source-connector
showing spreadsheetId is not a recognized parameter: