espeed / bulbs

A Python persistence framework for graph databases like Neo4j, OrientDB and Titan.
http://bulbflow.org
Other
622 stars 83 forks source link

Rexster create edge error with OrientDB #122

Closed yiziz closed 10 years ago

yiziz commented 10 years ago

I'm getting errors when I try to create an edge using bulb and rexter with orientDB.

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/bulbs/element.py", line 879, in create
    resp = self.client.create_edge(outV, label, inV, data, keys=_keys)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 454, in create_edge
    return self.create_indexed_edge(outV,label,inV,data,index_name,keys=keys)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 990, in create_indexed_edge
    resp = self.gremlin(script,params)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 356, in gremlin
    return self.request.post(gremlin_path, params)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rest.py", line 131, in post
    return self.request(POST, path, params)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rest.py", line 186, in request
    return self.response_class(http_resp, self.config)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 198, in __init__
    self.handle_response(response)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 222, in handle_response
    response_handler(http_resp)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rest.py", line 50, in server_error
    raise SystemError(http_resp)
SystemError: ({'status': '500', 'transfer-encoding': 'chunked', 'server': 'grizzly/2.2.16', 'connection': 'close', 'date': 'Sat, 25 Jan 2014 00:09:22 GMT', 'access-control-allow-origin': '*', 'content-type':
 'application/json'}, '{"message":"","error":"javax.script.ScriptException: java.lang.ClassCastException: com.orientechnologies.orient.core.id.ORecordId cannot be cast to com.orientechnologies.orient.core.re
cord.ORecord","api":{"description":"evaluate an ad-hoc Gremlin script for a graph.","parameters":{"rexster.returnKeys":"an array of element property keys to return (default is to return all element propertie
s)","rexster.showTypes":"displays the properties of the elements with their native data type (default is false)","load":"a list of \'stored procedures\' to execute prior to the \'script\' (if \'script\' is n
ot specified then the last script in this argument will return the values","rexster.offset.end":"end index for a paged set of data to be returned","rexster.offset.start":"start index for a paged set of data
to be returned","params":"a map of parameters to bind to the script engine","language":"the gremlin language flavor to use (default to groovy)","script":"the Gremlin script to be evaluated"}},"success":false
}')

For more info: Rexster create edge error with OrientDB

espeed commented 10 years ago

See http://stackoverflow.com/questions/21341349/rexster-create-edge-error-with-orientdb

yiziz commented 10 years ago

Curl statments

admin@sys:~$ curl -X POST  -H "Accept: Application/json" -H "Content-Type: application/json" 
http://localhost:8182/graphs/orientdb/tp/gremlin -d '{"params": {"label_var": "label", "outV": "#9:0", "keys": null, 
"data": {}, "inV": "#9:1", "index_name": "edge", "label": "curltest"}, "script": "def createIndexedEdge = {\n    index 
= g.idx(index_name)\n    edge = g.addEdge(g.v(outV),g.v(inV),label)\n    for (entry in data.entrySet()) {\n      if 
(entry.value == null) continue;\n      edge.setProperty(entry.key,entry.value)\n      if (keys == null || 
keys.contains(entry.key))\n\tindex.put(entry.key,String.valueOf(entry.value),edge)\n    }\n    
index.put(label_var,String.valueOf(label),edge)\n    return edge\n  }\n  def transaction = { final Closure closure ->\n    
try {\n      results = closure();\n      g.commit();\n      return results; \n    } catch (e) {\n      g.rollback();\n      throw 
e;\n    }\n  }\n  return transaction(createIndexedEdge);"}'}

{"message":"","error":"javax.script.ScriptException: java.lang.ClassCastException: 
com.orientechnologies.orient.core.id.ORecordId cannot be cast to 
com.orientechnologies.orient.core.record.ORecord","api":{"description":"evaluate an ad-hoc Gremlin script for a 
graph.","parameters":{"rexster.returnKeys":"an array of element property keys to return (default is to return all 
element properties)","rexster.showTypes":"displays the properties of the elements with their native data type 
(default is false)","load":"a list of 'stored procedures' to execute prior to the 'script' (if 'script' is not specified then 
the last script in this argument will return the values","rexster.offset.end":"end index for a paged set of data to be 
returned","rexster.offset.start":"start index for a paged set of data to be returned","params":"a map of parameters 
to bind to the script engine","language":"the gremlin language flavor to use (default to groovy)","script":"the 
Gremlin script to be evaluated"}},"success":false}admin@sys:~$ 

http://localhost:8182/graphs/orientdb/edges

{"version":"2.5.0-SNAPSHOT","results":[],"totalSize":0,"queryTime":0.936274}

http://localhost:8182/graphs/orientdb/vertices

{"version":"2.5.0-SNAPSHOT","results":[{"name":"foo","_id":"#9:0","_type":"vertex"},{"name":"bar","_id":"#9:1","_type":"vertex"},{"name":"foo","_id":"#9:2","_type":"vertex"},{"name":"bar","_id":"#9:3","_type":"vertex"}],"totalSize":4,"queryTime":3.464395}

Also get this when I was deleting earlier, not sure if related

>>> g.edges.delete("18:0")
DELETE url:  http://localhost:8182/graphs/orientdb/edges/18%3A0
DELETE url:  http://localhost:8182/graphs/orientdb/edges/18%3A0
DELETE url:  http://localhost:8182/graphs/orientdb/edges/18%3A0
DELETE body: None
DELETE body: None
DELETE body: None
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/bulbs/element.py", line 952, in delete
    return self.client.delete_edge(_id)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 512, in delete_edge
    return self.request.delete(path, params=None)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rest.py", line 146, in delete
    return self.request(DELETE, path, params)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rest.py", line 186, in request
    return self.response_class(http_resp, self.config)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 198, in __init__
    self.handle_response(response)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rexster/client.py", line 222, in handle_response
    response_handler(http_resp)
  File "/usr/local/lib/python2.7/dist-packages/bulbs/rest.py", line 50, in server_error
    raise SystemError(http_resp)
SystemError: ({'status': '500', 'transfer-encoding': 'chunked', 'server': 'grizzly/2.2.16', 'connection': 'close', 'date': 'Mon, 27 Jan 2014 19:04:15 GMT', 'access-control-allow-origin': '*', 'content-type':
 'application/json;charset=UTF-8'}, '{"message":"Transaction failed on DELETE of edge.","error":"Database instance is not set in current thread. Assure to set it with: ODatabaseRecordThreadLocal.INSTANCE.set
(db);"}')
yiziz commented 10 years ago

Typed create_indexed_edge line by line into the rexster gremlin console, and I think I've isolated the error:

gremlin> index.put("label", String.valueOf("curltest"), edge)
==>javax.script.ScriptException: java.lang.ClassCastException: com.orientechnologies.orient.core.id.ORecordId 
cannot be cast to com.orientechnologies.orient.core.record.ORecord
espeed commented 10 years ago

See https://github.com/tinkerpop/rexster/issues/341

espeed commented 10 years ago

If you want to autoindex vertices but not edges, we can add a Bulbs Config option for autoindex_edge = True/False. Right now there is a Config option for autoindex = True/False, but this includes both vertices and edges.

yiziz commented 10 years ago

I think this would be useful, but I also think it was orientDB's edges that were throwing me off.

They really don't have much docs on it, but from what I gathered thus far:

  1. You can alter databases to use(or not use) lightweight edges
  2. Lightweight edges cannot be indexed
  3. Physical edges are created when attributes are set. i.e. edges aren't simply labeled
  4. Inverse of 3: lightweight edges are created when edges don't have attributes

Thus, when using bulbs and not fully understanding the context of orientDB's edges could get you in trouble:

Note: using orientDB 1.7

when auto index is off:

e = g.edges.create(g.V[0], "foo", g.V[1]) # works, gives 1 lightweight edge
e = g.edges.create(g.V[0], "foo", g.V[1], weight=1) # works, gives 1 physical edge

when auto index is on:

e = g.edges.create(g.V[0], "foo", g.V[1]) # doesn't work, errors out (possibly did create lightweight edge, but errored on index creation)
e = g.edges.create(g.V[0], "foo", g.V[1], weight=1) # works, gives 1 physical edge

This is what I gathered, but I'm not 100% confident on what exactly happened in the background of those statements. So the autoindex_edge would probably be a useful tool especially when working with orientDB

espeed commented 10 years ago

Do you know if OrientDB is going to stick with this design?

Titan started off not having support for indexing edges, but now it does and it doesn't distinguish between "lightweight" and "physical" edges AFAIK.

yiziz commented 10 years ago

Lightweight edges got supported in 1.4 on June 08, 2013. See releases

They are supported in 1.6.4 (current) and will be supported in 1.7, which will be released in a few weeks/months. I haven't read anything about dropping lightweight edges.

But lightweight edges are not simply physical edges that are not indexed. Lightweight edges were included for performance and they cannot be retrieved by g.E after creation.

Like I stated before, there isn't much documentation on lightweight edges, so I'm not sure what exactly they are, how best they should be used, or what problem their inclusion solves (I assume performance bottlenecks). Ivca noted to me that OrientDB's biggest bottleneck is massive edge insertion.

I think you'll have to ask Ivca about this, I think he's the CEO of Orient Technologies.

espeed commented 10 years ago

@lvca Are lightweight edges similar to Titan's original edge model?

lvca commented 10 years ago

@espeed I don't know Titan's edge model. Do you have any links to documentation about it?

espeed commented 10 years ago

@lvca Matthias recently posted a Titan data model doc: https://github.com/thinkaurelius/titan/wiki/Titan-Data-Model