search

neo4j / neo4j-python-driver

Neo4j Bolt driver for Python
https://neo4j.com/docs/api/python-driver/current/
Other
912 stars 186 forks source link

Rolling back transaction leads to AttributeError #266

Closed istrel closed 6 years ago

istrel commented 6 years ago

Prerequisites:

When we shutdown leader, we start facing such stack traces (... replaced parts of stack in our code);

OSError: [Errno 113] No route to host
  ...
  File "/code/db/neo4j.py", line 22, in read_transaction
    return self.session.read_transaction(*args, **kwargs)
  File "neo4j/__init__.py", line 697, in read_transaction
    return self._run_transaction(READ_ACCESS, unit_of_work, *args, **kwargs)
  File "neo4j/__init__.py", line 664, in _run_transaction
    self._open_transaction(access_mode, metadata, timeout)
  File "neo4j/__init__.py", line 610, in _open_transaction
    self._connect(access_mode)
  File "neo4j/__init__.py", line 367, in _connect
    self._connection = self._acquirer(access_mode)
  File "neobolt/routing.py", line 425, in acquire
    connection = self.acquire_direct(address)  # should always be a resolved address
  File "neobolt/direct.py", line 575, in acquire_direct
    connection = self.connector(address, error_handler=self.connection_error_handler)
  File "neo4j/__init__.py", line 255, in connector
    return connect(address, **dict(config, **kwargs))
  File "neobolt/direct.py", line 937, in connect
    raise last_error
  File "neobolt/direct.py", line 927, in connect
    s = _connect(resolved_address, **config)
  File "neobolt/direct.py", line 793, in _connect
    s.connect(resolved_address)
AttributeError: 'NoneType' object has no attribute 'rollback'
  ...
  File "neo4j/__init__.py", line 358, in __exit__
    self.close()
  File "neo4j/__init__.py", line 391, in close
    self.rollback_transaction()
  File "neo4j/__init__.py", line 643, in rollback_transaction
    self._connection.rollback(on_success=metadata.update)

Here is some logs from neobolt logger captured immediately before exception:

01:28:56 [#B50A]  C: RESET
01:28:56 [#B50A]  S: IGNORED
01:28:56 [#B50A]  S: SUCCESS {}
01:28:57 [#0000]  C: <RESOLVE> Address(host='10.230.195.3', port=7687)
01:28:57 [#0000]  C: <OPEN> ('10.230.195.3', 7687)
01:29:00 [#0000]  C: <ERROR> OSError 113 'No route to host'
01:29:00 [#0000]  C: <CLOSE> ('10.230.195.3', 7687)

Python libraries: neo4j==1.7.0 neobolt==1.7.0

Neo4j version: 3.4.7

technige commented 6 years ago

Thanks for reporting. This looks like a bug.

technige commented 6 years ago

@istrel Do you have any sample code that could help us reproduce the error?

istrel commented 6 years ago

@technige I've reduced to this sample script

import os
from neo4j.v1 import GraphDatabase, basic_auth

neo4j_addr = os.getenv("NEO4J_ADDR", "bolt+routing://<host>:7687")
neo4j_password = os.getenv("NEO4J_PASSWORD", "neo4j")

driver = GraphDatabase.driver(
    neo4j_addr,
    auth=basic_auth("neo4j", neo4j_password),
)

def do_iteration():
    with driver.session() as session:
        def increment_counter(tx):
            tx.run("""
                MERGE
                    (t:TestNode)
                ON CREATE SET
                    t.counter = 0
                SET
                    t.counter = t.counter + 1
            """, {})

        session.write_transaction(increment_counter)

for i in range(100000):
    print(i)
    do_iteration()

When I shut down leader I see the following:

...
500
501
Traceback (most recent call last):
  File "test.py", line 25, in do_iteration
    session.write_transaction(increment_counter)
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 701, in write_transaction
    return self._run_transaction(WRITE_ACCESS, unit_of_work, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 664, in _run_transaction
    self._open_transaction(access_mode, metadata, timeout)
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 610, in _open_transaction
    self._connect(access_mode)
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 367, in _connect
    self._connection = self._acquirer(access_mode)
  File "/usr/local/lib/python3.6/site-packages/neobolt/routing.py", line 425, in acquire
    connection = self.acquire_direct(address)  # should always be a resolved address
  File "/usr/local/lib/python3.6/site-packages/neobolt/direct.py", line 575, in acquire_direct
    connection = self.connector(address, error_handler=self.connection_error_handler)
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 255, in connector
    return connect(address, **dict(config, **kwargs))
  File "/usr/local/lib/python3.6/site-packages/neobolt/direct.py", line 937, in connect
    raise last_error
  File "/usr/local/lib/python3.6/site-packages/neobolt/direct.py", line 927, in connect
    s = _connect(resolved_address, **config)
  File "/usr/local/lib/python3.6/site-packages/neobolt/direct.py", line 793, in _connect
    s.connect(resolved_address)
OSError: [Errno 51] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 30, in <module>
    do_iteration()
  File "test.py", line 25, in do_iteration
    session.write_transaction(increment_counter)
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 358, in __exit__
    self.close()
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 391, in close
    self.rollback_transaction()
  File "/usr/local/lib/python3.6/site-packages/neo4j/__init__.py", line 643, in rollback_transaction
    self._connection.rollback(on_success=metadata.update)
AttributeError: 'NoneType' object has no attribute 'rollback'
technige commented 6 years ago

Fixed by https://github.com/neo4j/neo4j-python-driver/pull/271

istrel commented 6 years ago

Tested 1.7.1 - it works! 🎉

Thank you very much!

mhelal commented 4 years ago

Hi

I did not understand how it was fixed and what to do to recover from this error. Can you please explain more.

I am getting null connection trying to connect from neomodel for Django (https://pypi.org/project/django_neomodel/):

$ python3 manage.py install_labels Setting up indexes and constraints...

Found django_neomodel.DjangoNode ! Skipping class django_neomodel.DjangoNode is abstract Found qApp.models.Surah

Thank you