AttributeError: AsyncGraphTraversal.last_traverser

[grvhi]

I'm sure I'm missing a step somewhere, but I can't immediately establish what I'm doing wrong from the docs. Here is my sample code to reproduce the problem, followed by the traceback and the output from pip freeze. Any tips gratefully received!

    import asyncio

    from goblin import Goblin

    class City(goblin.Vertex):
        value = goblin.Property(goblin.String)

    loop = asyncio.get_event_loop()
    app = loop.run_until_complete(
        Goblin.open(loop, hosts=['neptune.opendna.internal'])
    )
    app.register(City)
    session = loop.run_until_complete(app.session())  # type: Session

    async def save_location(obj):
        session.add(obj)
        return await session.flush()

    city = City()
    city.value = 'Cape Town'
    loop.run_until_complete(save_location(city))

Traceback (most recent call last):
  File "/Users/george/sites/odna-neptune/vesta/vertices/location.py", line 54, in <module>
    loop.run_until_complete(save_location(city))
  File "/Users/george/.pyenv/versions/3.6.0/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/Users/george/sites/odna-neptune/vesta/vertices/location.py", line 50, in save_location
    return await session.flush()
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/goblin/session.py", line 242, in flush
    await self.save(elem)
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/goblin/session.py", line 292, in save
    result = await self.save_vertex(elem)
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/goblin/session.py", line 311, in save_vertex
    self._update_vertex)
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/goblin/session.py", line 412, in _save_element
    result = await create_func(elem)
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/goblin/session.py", line 419, in _add_vertex
    return await self._add_properties(traversal, props, vertex)
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/goblin/session.py", line 473, in _add_properties
    return await self._simple_traversal(traversal, elem)
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/goblin/session.py", line 388, in _simple_traversal
    elem = await traversal.next()
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/aiogremlin/process/graph_traversal.py", line 56, in next
    return await self.__anext__()
  File "/Users/george/sites/odna-neptune/venv/lib/python3.6/site-packages/aiogremlin/process/graph_traversal.py", line 18, in __anext__
    object = self.last_traverser.object
AttributeError: 'str' object has no attribute 'object'

aenum==1.4.5
aiogremlin==3.2.5
aiohttp==1.3.3
async-timeout==2.0.0
chardet==3.0.4
Cython==0.27.3
goblin==2.1.0
gremlinpython==3.3.1
idna==2.6
inflection==0.3.1
multidict==3.3.2
PyYAML==3.12
six==1.10.0
tornado==4.5.2
ujson==1.35
uvloop==0.9.1
yarl==0.9.8

[grvhi]

A bit more information:

• When the exception is raised, self.last_traverser is a str with the value @type.
• We're using the new AWS Neptune database (currently in preview)

Is it possible that the response format is slightly different, causing aiogremlin to return the first key (@type) of the response from the database, rather than the full object?

[davebshow]

Sorry for the slow reply. Looks like this is a version problem. Aiogremlin/Goblin have not yet been updated for the latest version of TP. I would try rebuilding your environment using these installs:

$ pip install gremlinpython==3.2.6 --no-deps
$ pip install aiogremlin==3.2.6
$ pip install goblin

[davebshow]

Installing is a bit weird at this point to avoid getting Tornado. This should be fixed soon.

[davebshow]

Also, I am not sure which version TP Neptune is using. If it is the 3.3 line, there may be some issues with aiogremlin/goblin. I am hoping to get these libraries updated with different versions for the 3.2 and 3.3 lines, but I have been really busy lately.

[grvhi]

No problem David - thanks for getting back to me. I've done as you suggested and installed the versions listed above. I now get this error:

aiogremlin.exception.GremlinServerError: 500: Properties on a vertex property is not supported

However, if I install goblin==2.0.0, then everything works perfectly. I'm going to try to extract the serialiser used by default in that version and use it in the latest versions to see if that's where the problem lies. But if you have any other thoughts, I'd be very grateful!

Thanks again

BTW - PyPi only has aiogremlin==3.2.6rc1

[davebshow]

Huh. Does Neptune not support metaproperties? Seems like it doesn't. There were a lot of improvements around metaproperties and optimization of related queries between Goblin 2.0.0 and 2.1.0, so I assume that is the issue. It is a bit hard for me to troubleshoot this, as I don't believe there is public/free access to a Neptune endpoint I could test against.

[grvhi]

Ok thanks David. You're correct: Neptune does not support metaproperties. Presumably this limits us to using goblin 2.0.0? Would a PR to allow for this limitation (via a config variable) in the current version of goblin be feasible/acceptable?

Although Neptune is not currently public, I don't think it will be long before it is, so I imagine others will look to use goblin with Neptune as well.

[davebshow]

Yes, this should be dealt with, as there also may be other providers that don't support metaprops as well. Both Goblin and aiogremlin need to be updated in general. I've been super busy lately, and my current job doesn't really have anything to do with this. I am really going to try to do some work on these libraries though in the next week or so.

I am happy to accept PRs. For a big change like this, maybe we should open a separate issue to discuss the implementation you propose. If my memory serves, I think all of the offending traversals will be in goblin.session, this one in particular.

[grvhi]

Probably sensible. However, it may be a couple of weeks before we'll be able to make any meaningful contributions. We are working on our Neptune POC though (of which Goblin is a part), so we'll try to extract a useful proposal as we work through our first hypotheses.

[davebshow]

Quick update here @grvhi. Yesterday I released aiogremlin 3.2.6, 3.2.7, 3.3.0, 3.3.1. You should be able to use 3.3.0 to talk to Neptune. Also, I think I may have some time this weekend to look at Goblin. I plan on adding two features:

1. Disable metaprops
2. Compat with any version of TinkerPop (basically you will have to pass the traversal source/remote connection to Goblin).

Since I don't have access to Neptune, it would be great if you could try out the new version of Goblin when available. I will upload a release candidate to PyPi when it is ready.

[grvhi]

That great, thanks @davebshow - we’ll get on it ASAP and let you know how we get on. Thanks very much for the quick turnaround: much appreciated!

[davebshow]

Haven't gotten to this yet. Been crazy busy. On my list though.

[mbbfyb]

I am not quite clear what the exact python packages you are referencing here, but this is my setup that I finally got working after reading the above and using Neptune:

pip freeze

aenum==1.4.5
aiogremlin==3.2.4
aiohttp==1.3.3
async-timeout==2.0.0
chardet==3.0.4
goblin==2.0.0
gremlinpython==3.3.1
idna==2.6
idna-ssl==1.0.0
inflection==0.3.1
multidict==4.1.0
pkg-resources==0.0.0
PyYAML==3.12
six==1.10.0
tornado==4.5.3
yarl==0.9.8

python console:

import goblin
import asyncio
from goblin import Goblin

loop = asyncio.get_event_loop()
app = loop.run_until_complete(Goblin.open(loop, hosts=["<neptune_host>"]))

class Person(goblin.Vertex):
    name = goblin.Property(goblin.String)

class City(goblin.Vertex):
    name = goblin.Property(goblin.String)
    population = goblin.Property(goblin.Integer)

class BornIn(goblin.Edge):
    pass

leif = Person()
leif.name = 'Leif'
detroit = City()
detroit.name = 'Detroit'
detroit.population = 5311449
leif_born_in_detroit = BornIn(leif, detroit)

app.register(Person, City, BornIn)
async def create_elements(app):
    session = await app.session()
    session.add(leif, detroit, leif_born_in_detroit)
    await session.flush()

loop.run_until_complete(create_elements(app))

The command above completes, and my 2 nodes and edge are added to the graph, but when I exit the console I get the following traceback

Task was destroyed but it is pending!
task: <Task pending coro=<Connection._receive() running at /home/ubuntu/neptune/neptune_venv/lib/python3.5/site-packages/aiogremlin/driver/connection.py:161> wait_for=<Future pending cb=[Task._wakeup()]>>
Task was destroyed but it is pending!
task: <Task pending coro=<Connection._receive() running at /home/ubuntu/neptune/neptune_venv/lib/python3.5/site-packages/aiogremlin/driver/connection.py:161> wait_for=<Future pending cb=[Task._wakeup()]>>
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f45a7ad9da0>
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f45a7aa1278>
Unclosed response
client_response: <ClientResponse(ws://<neptune>:8182/gremlin) [101 Switching Protocols]>
<CIMultiDictProxy('Upgrade': 'websocket', 'Connection': 'upgrade', 'Sec-Websocket-Accept': 'ZWJSTnUYZkqnhmjUWbvZHyrhhmk=')>

Unclosed response
client_response: <ClientResponse(ws://<neptune>:8182/gremlin) [101 Switching Protocols]>
<CIMultiDictProxy('Upgrade': 'websocket', 'Connection': 'upgrade', 'Sec-Websocket-Accept': 'X448MJhU1et/SBiF8KvBMQoHcBE=')>

I figure I am missing something with app.close() or app.stop() (I am not very familiar with asyncio), but I can wait a fairly significant time and always get that response.

EDIT 2/16/18: Figured out how to properly close the app via

async def close(app):
    await app.close()

loop.run_until_complete(close(app))
loop.close()

So, everything seems to be working cleanly.

[kkom]

Hey @davebshow – sorry for the issue necromancy, but I figured it would be better to keep the original context!

I am testing the following package versions with Neptune 1.0.2.2 (see: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-differences.html):

aiogremlin==3.3.5
goblin==2.2.3
gremlinpython==3.4.3

aiogremlin includes a fix from https://github.com/goblin-ogm/aiogremlin/pull/2 to enable SSL connection, hence the unreleased version 3.3.5.

I am stilling running into the same problem when defining this kind of object:

class Person(element.Vertex):
    __label__ = 'person'
    name = properties.Property(properties.String)
    age = properties.Property(properties.Integer)

Trying to add a vertex results in this error:

GremlinServerError: 499: {"requestId":"3e83b817-e993-409c-afc0-693a676e533b","code":"UnsupportedOperationException","detailedMessage":"Properties on a vertex property is not supported"}

I was just playing with the object schema, and after changing it to:

class Person(element.Vertex):
    __label__ = 'person'
    name = properties.String
    age = properties.Integer

Adding the vertex succeeds, but I think the property is not stored on the graph. At least I can't see it even if using the official gremlin console...

Traversing the graph in this way results in the same exception:

async def execute_person_traversal(
    session: Session,
) -> List[element.GenericVertex]:
    vertices = []

    async for vertex in session.traversal(Person):
        vertices.append(vertex)

    return vertices

GremlinServerError: 500: 499: {"requestId":"47fa052d-8602-4d25-b0e0-6f51d009bd0e","code":"UnsupportedOperationException","detailedMessage":"Properties on a vertex property is not supported"}

And that's regardless of how I define the object. (I previously could get it to work somehow, but I don't remember how. Maybe it was related to the state of the graph? Probably not though...)

Anyway, I would like to find out if it's possible to reconfigure Goblin to support Neptune. And if not, how much of an effort it would be to do so?

Also, what kind of limitations would that introduce in general? Maybe Neptune is so limited, that it's not even worth considering it as a practical Gremlin-backed graph database?

Thanks in advance for any help!

Best, Konrad

[kkom]

FYI, I reposted this in the new goblin repo here: https://github.com/goblin-ogm/goblin/issues/1