rmoff
commented
4 years ago Is this relating to a particular demo in this repository?
Closed erdelacruz closed 4 years ago
rmoff
commented
4 years ago Is this relating to a particular demo in this repository?
erdelacruz
commented
4 years ago Hi rmoff,
I used the docker-compose.yml from demo-scene/community-components-only/docker-compose.yml as my base yml file, i just added an image for confluent-control-center and configure a build setting to use dockerfile for kafka-connect image, so that other connectors such salesforce will be installed.
I was able to build and deploy the services without issue and able to access to connectors in control center, however after configuring a Salesforce Connector in control center, the connector won't deploy due to an error TimeoutException: Timed out waiting for a node assignment
I'm hoping you can help me with my issue. Thank you. Regards, Rico
rmoff
commented
4 years ago Hi, I'm going to close this here because it's not related to one of the demos. The best place for community support is on:
→ Slack group: http://cnfl.io/slack or → Mailing list: https://groups.google.com/forum/#!forum/confluent-platform
Thanks :)
Hello, i'm currently getting an error org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment in the connector status after configuring a Salesforce Source Connector in Confluent Center > Connect Page.
This is my docker-compose.yml file:
`
version: '2' services: zookeeper: image: confluentinc/cp-zookeeper:latest ports:
'31000:31000' environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 KAFKA_JMX_HOSTNAME: "localhost" KAFKA_JMX_PORT: 31000
kafka:
"
-._,-'"-.,-'"`-.,-'"-._,-'"-.,-'"`-.,-'"-._,-'"-.,-'"`-.,-An important note about accessing Kafka from clients on other machines:
-----------------------------------------------------------------------
#
The config used here exposes port 9092 for external connections to the broker
i.e. those from outside the docker network. This could be from the host machine
running docker, or maybe further afield if you've got a more complicated setup.
If the latter is true, you will need to change the value 'localhost' in
KAFKA_ADVERTISED_LISTENERS to one that is resolvable to the docker host from those
remote clients
#
For connections internal to the docker network, such as from other services
and components, use kafka:29092.
#
See https://rmoff.net/2018/08/02/kafka-listeners-explained/ for details
"
-._,-'"-.,-'"`-.,-'"-._,-'"-.,-'"`-.,-'"-._,-'"-.,-'"`-.,-image: confluentinc/cp-enterprise-kafka:5.3.0 ports:
zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true" KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 100 CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: kafka:29092 CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT: zookeeper:2181 CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1 CONFLUENT_METRICS_ENABLE: 'false' CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous' KAFKA_JMX_HOSTNAME: "localhost" KAFKA_JMX_PORT: 31001
schema-registry: image: confluentinc/cp-schema-registry:latest depends_on:
'31002:31002' environment: SCHEMA_REGISTRY_HOST_NAME: schema-registry SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181 SCHEMA_REGISTRY_JMX_HOSTNAME: "localhost" SCHEMA_REGISTRY_JMX_PORT: 31002
ksql-server: image: confluentinc/cp-ksql-server:latest depends_on:
'8088:8088' environment: KSQL_BOOTSTRAP_SERVERS: kafka:29092 KSQL_LISTENERS: http://0.0.0.0:8088 KSQL_KSQL_SCHEMA_REGISTRY_URL: http://schema-registry:8081 KSQL_KSQL_SERVICE_ID: asgard KSQL_JMX_HOSTNAME: "localhost" KSQL_JMX_PORT: 31003 KSQL_LOG4J_ROOT_LOGLEVEL: "INFO" KSQL_LOG4J_LOGGERS: "org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR"
--- Processing log config ---
KSQL_LOG4J_PROCESSING_LOG_BROKERLIST: kafka:29092 KSQL_LOG4J_PROCESSING_LOG_TOPIC: asgard_processing_log KSQL_KSQL_LOGGING_PROCESSING_TOPIC_NAME: asgard_processing_log KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true" KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
ksql-cli: image: confluentinc/cp-ksql-cli:latest depends_on:
ksql-server entrypoint: /bin/sh tty: true
kafka-connect: image: confluentinc/cp-kafka-connect:latest ports:
schema-registry
Uncomment this bit if you want to automatically install the datagen and twitter connectors
and set the datagen connector running
command:
- bash
- -c
- |
confluent-hub install --no-prompt confluentinc/kafka-connect-datagen:0.1.1
confluent-hub install --no-prompt jcustenborder/kafka-connect-twitter:0.2.32
/etc/confluent/docker/run &
echo "Waiting for Kafka Connect to start listening on kafka-connect ⏳"
while [ $$(curl -s -o /dev/null -w %{http_code} http://kafka-connect:8083/connectors) -ne 200 ] ; do
echo -e $$(date) " Kafka Connect listener HTTP state: " $$(curl -s -o /dev/null -w %{http_code} http://kafka-connect:8083/connectors) " (waiting for 200)"
sleep 5
done
nc -vz kafka-connect 8083
echo -e "\n--\n+> Creating Connector"
curl -X POST http://kafka-connect:8083/connectors -H "Content-Type: application/json" -d '
{
"name": "datagen-pageviews",
"config": {
"connector.class": "io.confluent.kafka.connect.datagen.DatagenConnector",
"kafka.topic": "pageviews",
"quickstart": "pageviews",
"max.interval": 100,
"iterations": 10000000,
"tasks.max": "1"
}
}
'
sleep infinity
rest-proxy: image: confluentinc/cp-kafka-rest:latest depends_on:
'31005:31005' environment: KAFKA_REST_HOST_NAME: rest-proxy KAFKA_REST_BOOTSTRAP_SERVERS: 'kafka:29092' KAFKA_REST_LISTENERS: "http://0.0.0.0:8082" KAFKA_REST_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081' KAFKAREST_JMX_HOSTNAME: "localhost" KAFKAREST_JMX_PORT: 31005
control-center: image: confluentinc/cp-enterprise-control-center:latest hostname: control-center container_name: control-center depends_on: