amundsen-io / amundsen

Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data.
https://www.amundsen.io/amundsen/
Apache License 2.0
4.37k stars 953 forks source link

Bug Report neo4j.exceptions.ServiceUnavailable: Couldn't connect to localhost:7687 trying to load sample data #2181

Open georgepoly-maker opened 1 year ago

georgepoly-maker commented 1 year ago

/Users/.../workspace/amundsen/amundsen/databuilder/example/scripts/sample_data_loader.py:64: MovedIn20Warning: The declarative_base() function is now available as sqlalchemy.orm.declarative_base(). (deprecated since: 2.0) (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9) Base = declarative_base() Traceback (most recent call last): File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/_socket.py", line 303, in connect return BoltSocket._handshake(s, resolved_address, deadline) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/_socket.py", line 239, in _handshake raise ServiceUnavailable( neo4j.exceptions.ServiceUnavailable: Connection to [::1]:7687 closed without handshake response

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/.../workspace/amundsen/amundsen/databuilder/example/scripts/sample_data_loader.py", line 357, in run_table_column_job('example/sample_data/sample_table.csv', 'example/sample_data/sample_col.csv') File "/Users/.../workspace/amundsen/amundsen/databuilder/example/scripts/sample_data_loader.py", line 163, in run_table_column_job job.launch() File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/amundsen_databuilder-7.4.4-py3.11.egg/databuilder/job/job.py", line 76, in launch File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/amundsen_databuilder-7.4.4-py3.11.egg/databuilder/job/job.py", line 72, in launch File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/amundsen_databuilder-7.4.4-py3.11.egg/databuilder/publisher/base_publisher.py", line 40, in publish File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/amundsen_databuilder-7.4.4-py3.11.egg/databuilder/publisher/base_publisher.py", line 37, in publish File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/amundsen_databuilder-7.4.4-py3.11.egg/databuilder/publisher/neo4j_csv_publisher.py", line 229, in publish_impl File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/amundsen_databuilder-7.4.4-py3.11.egg/databuilder/publisher/neo4j_csv_publisher.py", line 276, in _create_indices File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/amundsen_databuilder-7.4.4-py3.11.egg/databuilder/publisher/neo4j_csv_publisher.py", line 519, in _try_create_index File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/work/simple.py", line 205, in run self._connect(self._config.default_access_mode) File "/Users/l.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/work/simple.py", line 108, in _connect super()._connect(access_mode) File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/work/init.py", line 82, in _connect self._connection = self._pool.acquire( ^^^^^^^^^^^^^^^^^^^ File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/init.py", line 925, in acquire return self._acquire(self.address, deadline) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/init.py", line 800, in _acquire return connection_creator() ^^^^^^^^^^^^^^^^^^^^ File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/init.py", line 730, in connection_creator connection = self.opener(address, deadline) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/init.py", line 906, in opener return Bolt.open( ^^^^^^^^^^ File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/init.py", line 326, in open s, pool_config.protocol_version, handshake, data = BoltSocket.connect( ^^^^^^^^^^^^^^^^^^^ File "/Users/.../workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/io/_socket.py", line 327, in connect raise ServiceUnavailable( neo4j.exceptions.ServiceUnavailable: Couldn't connect to localhost:7687 (resolved to ('[::1]:7687', '127.0.0.1:7687')): Connection to [::1]:7687 closed without handshake response Connection to 127.0.0.1:7687 closed without handshake response

15:24 File "/Users/xxxxxxx/workspace/amundsen/amundsen/databuilder/venv/lib/python3.11/site-packages/neo4j/work/simple.py", line 205, in run self._connect(self._config.default_access_mode)

Expected Behavior

I followed the demo step by step i should be able to load the data

Current Behavior

<

Steps to Reproduce

1.Make sure you have at least 3GB of disk space available to Docker. Install docker and docker-compose. Clone this repo and its submodules by running: $ git clone --recursive https://github.com/amundsen-io/amundsen.git Enter the cloned directory and run the command below:

For Neo4j Backend

$ docker-compose -f docker-amundsen.yml up

For Atlas

$ docker-compose -f docker-amundsen-atlas.yml up If it’s your first time, you may want to proactively go through troubleshooting steps, especially the first one related to heap memory for ElasticSearch and Docker engine memory allocation (leading to Docker error 137). Ingest provided sample data into Neo4j by doing the following: (Please skip if you are using Atlas backend)

In a separate terminal window, change directory to databuilder.

sample_data_loader python script included in examples/ directory uses elasticsearch client, pyhocon and other libraries. Install the dependencies in a virtual env and run the script by following the commands below. See Windows Troubleshooting if you encounter an error on python3 setup.py install regarding extas_require on windows. $ python3 -m venv venv $ source venv/bin/activate $ pip3 install --upgrade pip $ pip3 install -r requirements.txt $ python3 setup.py install $ python3 example/scripts/sample_data_loader.py 2. 3. 4.

boring-cyborg[bot] commented 1 year ago

Thanks for opening your first issue here!

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

martinroddam commented 11 months ago

Seeing the same error with Neo4j backend

koumi15cancer commented 9 months ago

Hi, I've tried simulated the error from your description:

  1. The first error is because _your container neo4jamundsen is exited.

I've checked the issue from both neo4j repository and find this one kind tricky but it seems the folder /var/lib/neo4j/conf could not be accessed or written inside container .

image

As for solution to solve this:

I replace the mount folder from: volumes: - ./example/docker/neo4j/conf:/var/lib/neo4j/conf to - ./example/docker/neo4j/conf:/conf

It works and the config file is written inside path var/lib/neo4j/conf .

I think this one it 's relevant to config file pattern from neo4j since they also pointed that the default location for config as outside one referenced from below link:

https://neo4j.com/docs/operations-manual/current/configuration/file-locations/?utm_source=google&utm_medium=PaidSearch&utm_campaign=GDB&utm_content=APAC-X-Awareness-GDB-Text&utm_term=&gad_source=1&gclid=CjwKCAiAx_GqBhBQEiwAlDNAZihhvFpWKNw1zPsF795zKKSiWFslH-6zc3cqFzkoMB0rqoW3IONFMxoCL_4QAvD_BwE Default config location path

Same issue https://github.com/neo4j/docker-neo4j/issues/461