pythongremlin errors

[esdesai]

When trying to connect to Neptune, getting the following error:

TypeError: 'LazyHttpHeaders' object is not iterable

----- Sample code below --- from neptune_python_utils.gremlin_utils import GremlinUtils

GremlinUtils.init_statics(globals())

gremlin_utils = GremlinUtils()

conn = gremlin_utils.remote_connection() g = gremlin_utils.traversal_source(connection=conn)

print(g.V().limit(10).valueMap().toList())

conn.close()

[beebs-systap]

@krlawrence Any ideas on this one?

[akash-codes93]

Hey, any updates on this one?

Stack Trace: File "/usr/local/lib/python3.7/site-packages/gremlin_python/process/traversal.py" in toList

58. return list(iter(self))

File "/usr/local/lib/python3.7/site-packages/gremlin_python/process/traversal.py" in next

48. self.traversal_strategies.apply_strategies(self)

File "/usr/local/lib/python3.7/site-packages/gremlin_python/process/traversal.py" in apply_strategies

573. traversal_strategy.apply(traversal)

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/remote_connection.py" in apply

149. remote_traversal = self.remote_connection.submit(traversal.bytecode)

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/driver_remote_connection.py" in submit

56. result_set = self._client.submit(bytecode, request_options=self._extract_request_options(bytecode))

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/client.py" in submit

127. return self.submitAsync(message, bindings=bindings, request_options=request_options).result()

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/client.py" in submitAsync

148. return conn.write(message)

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/connection.py" in write

55. self.connect()

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/connection.py" in connect

45. self._transport.connect(self._url, self._headers)

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/tornado/transport.py" in connect

41. lambda: websocket.websocket_connect(url, compression_options=self._compression_options))

File "/usr/local/lib/python3.7/site-packages/tornado/ioloop.py" in run_sync

576. return future_cell[0].result()

File "/usr/local/lib/python3.7/site-packages/tornado/ioloop.py" in run

547. result = func()

File "/usr/local/lib/python3.7/site-packages/gremlin_python/driver/tornado/transport.py" in

41. lambda: websocket.websocket_connect(url, compression_options=self._compression_options))

File "/usr/local/lib/python3.7/site-packages/tornado/websocket.py" in websocket_connect

1328. request.headers = httputil.HTTPHeaders(request.headers)

File "/usr/local/lib/python3.7/site-packages/tornado/httputil.py" in init

146. self.update(*args, **kwargs)

File "/usr/local/lib/python3.7/_collections_abc.py" in update

846. for key, value in other:

Exception Type: TypeError at //api/creategraph/ Exception Value: 'LazyHttpHeaders' object is not iterable

[domjancik]

Hi all, also ran into this when testing with the Gremlin / Python samples provided here https://docs.aws.amazon.com/neptune/latest/userguide/get-started-cfn-lambda.html

I assume it might be a version mismatch between the libraries here and what these utils were designed for.

But in any case, the fix is fairly simple and should retain the lazy nature of the headers, which I would again assume is there so the contents are fresh for each request.

Add the following lines to the LazyHttpHeaders class in endpoints.py to make the object iterable:

    def __iter__(self):
        return iter(self.items())

This will generate the headers and return their built-in iterator.

The full class then becomes

class LazyHttpHeaders:

    def __init__(self, lazy_headers):
        self.lazy_headers = lazy_headers

    def get_all(self) -> Iterable[Tuple[str, str]]:
        return self.items()

    def items(self):
        return self.lazy_headers().items()

    def __iter__(self):
        return iter(self.items())

For reference: https://www.w3schools.com/python/python_iterators.asp

[iansrobinson]

Thank you @domjancik for identifying the issue and proposing the fix – and pointing out that the Lambda CFN provided one way of testing. I was able to reproduce in a Lambda created by the CFN, and then incorporate the fix: https://github.com/awslabs/amazon-neptune-tools/pull/143

[domjancik]

Following up on this, while not exactly related to the original issue, we've had some problems with LazyHttpHeaders again when downgradinggremlinpython version to 3.5.6 due to some compatibility constraints.

This version tries to directly write headers, so the following error comes up:

LazyHttpHeaders' object does not support item assignment

In context:

{
    "levelname": "ERROR",
    "exc_info": "(<class 'TypeError'>, TypeError(\"'LazyHttpHeaders' object does not support item assignment\"), <traceback object at 0x7fd94083bcc0>)",
    "location": "gremlin_utils.py remote_connection:79",
    "message": "Failed to connect to Neptune.",
    "traceback": [
        "  File \"/var/task/functions/common/db/neptune_python_utils/gremlin_utils.py\", line 64, in remote_connection\n    connection = DriverRemoteConnection(\n",
        "  File \"/opt/python/gremlin_python/driver/driver_remote_connection.py\", line 67, in __init__\n    self._client = client.Client(url, traversal_source,\n",
        "  File \"/opt/python/gremlin_python/driver/client.py\", line 95, in __init__\n    self._fill_pool()\n",
        "  File \"/opt/python/gremlin_python/driver/client.py\", line 111, in _fill_pool\n    conn = self._get_connection()\n",
        "  File \"/opt/python/gremlin_python/driver/client.py\", line 145, in _get_connection\n    return connection.Connection(\n",
        "  File \"/opt/python/gremlin_python/driver/connection.py\", line 44, in __init__\n    self._headers[useragent.userAgentHeader] = useragent.userAgent\n"
    ]
}

Fix here is to allow some extra headers to be written using the __setitem__ python data model method:

class LazyHttpHeaders:
    def __init__(self, lazy_headers):
        self.lazy_headers = lazy_headers
        self.extra_headers = {}

    def get_all(self) -> Iterable[Tuple[str, str]]:
        return self.items()

    def items(self):
        combined_headers = {**self.lazy_headers(), **self.extra_headers}
        return combined_headers.items()

    def __iter__(self):
        return iter(self.items())

    def __setitem__(self, key, value):
        self.extra_headers[key] = value