search

neo4j-contrib / neo4j-streams

Neo4j Kafka Connector
https://neo4j.com/docs/kafka
Apache License 2.0
173 stars 71 forks source link

Local setup of Kafka to Neo4j by using Docker #533

Open maheshtati441 opened 2 years ago

maheshtati441 commented 2 years ago

Guidelines

Please note that GitHub issues are only meant for bug reports/feature requests. If you have questions on how to use Streams, please ask on StackOverflow instead of creating an issue here.

Expected Behavior (Mandatory)

When an event is generated from Kafka, the event should be updated in Neo4j by the cypher query defined in the Neo4j Sink Connector

Actual Behavior (Mandatory)

I couldn't see the connect clusters and the tab to create schema and unable to identify the issue.

How to Reproduce the Problem

  1. Run the docker-compose.yml file and the required images will be installed in the Docker and browse for localhost:9021
  2. By using the Maven clean install, able to get the zip file and placed under the Plugins folder where the docker-compose.yml is there.
  3. Check for the connect-plugins and able to see them through Postman but not in the connect under the control center (localhost:9021)
  4. Run the curl command to create the sink connector and check in the control center and it will not be found eventhough it got created. But it can be found by querying through Postman.
  5. Unable to find to create the schema under the Topic created(my-topic)

    Simple Dataset (where it's possibile)

//Insert here a set of Cypher statements that helps us to reproduce the problem

Steps (Mandatory)

  1. Started the docker image

2.Showing the docker containers in which Schema-Registry is in exited state after the docker run image

  1. In the Confluent (localhost:9021), under the connect there are no connectors found image

  2. If I request for connect-plugins through Postman, I can see the kafka connectors [ { "class": "org.apache.kafka.connect.mirror.MirrorCheckpointConnector", "type": "source", "version": "1" }, { "class": "org.apache.kafka.connect.mirror.MirrorHeartbeatConnector", "type": "source", "version": "1" }, { "class": "org.apache.kafka.connect.mirror.MirrorSourceConnector", "type": "source", "version": "1" }, { "class": "streams.kafka.connect.sink.Neo4jSinkConnector", "type": "sink", "version": "2.0.2" }, { "class": "streams.kafka.connect.source.Neo4jSourceConnector", "type": "source", "version": "2.0.2" } ]

  3. By using the curl command, I'm able to post and able to see the sink connectors through Postman image By using (localhost:8083/connectors/), I can see the sink connector getting created. image

  4. Under the Schema section of the Topic which is created, I don't see any option to create a schema. image

  5. If I query for the subjects and Topics I'm not getting any response. image

image

Screenshots (where it's possibile)

image image image image image image image image

Specifications (Mandatory)

Currently used versions

Versions

OneCricketeer commented 2 years ago

Firstly, I recommend not using Control Center. It will not help you debug the problem(s).

Your schema registry (port 8081) is simply down, as shown by your Docker Desktop image (and it's not clear you're using Avro data, anyway for the Connector). You should use docker-compose logs schema-registry to see why that could be (perhaps your machine doesn't have enough memory to run everything, and removing Control Center would be a step towards fixing that)

Also, port 9092 is not an HTTP address, so using POSTMan on it wont work.

mroiter-larus commented 2 years ago

Hi @maheshtati441,

this not seems a Neo4j Streams related issue. I suppose there is a misconfiguration in your docker-compose.yml. If you can share the docker-compose file we can try to help you.

maheshtati441 commented 1 year ago

Hi Mauro,

Thanks for your Email. Attaching the docker file for your reference. Awaiting for your response...!!!

Thank You, Mahesh tati

On Tue, Sep 20, 2022 at 8:44 PM Mauro Roiter @.***> wrote:

Hi @maheshtati441 https://github.com/maheshtati441,

this not seems a Neo4j Streams related issue. I suppose there is a misconfiguration in your docker-compose.yml. If you can share the docker-compose file we can try to help you.

— Reply to this email directly, view it on GitHub https://github.com/neo4j-contrib/neo4j-streams/issues/533#issuecomment-1252509390, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIP2GFTJXXDG4EQ6XGYA2PDV7HIFVANCNFSM5VY3SRKA . You are receiving this because you were mentioned.Message ID: @.***>

ali-ince commented 1 year ago

He @maheshtati441, is this still a problem and if so, would you please post the docker compose file into the github issue so that we can access it?