unipop-graph / unipop

Data Integration Graph
Apache License 2.0
204 stars 35 forks source link

Unipop graph.edge.count does not return correct result #115

Open baozengkai opened 6 years ago

baozengkai commented 6 years ago

Hi, My Elasticsearch has 3170000 documents. And I want to mapping this document to edge.My Elasticsearch documents like this: { "_index" : "company", "_type" : "data", "_id" : "AV9RS_qd695UKO_41Ei9", "_score" : 1.0, "_source" : { "time" : "January 02 2017 18:11:46.000", "document" : "XXX.docx", "user" : "XX" "userId" : XXX, "documentId" : XXX, "action" : "open", } I use tinkerpop-server+unipop to do the graph search.My unipop mapping json like this: ` { "class": "org.unipop.elastic.ElasticSourceProvider", "clusterName": "elasticsearch", "addresses": "http://localhost:9201", "vertices": [ { "index": "company", "id": "@userId", "label": "user", "properties": { "name": "@user" }, }, { "index": "company", "id": "@documentId", "label": "document", "properties": { "name": "@document" }, } ],

"edges": [ { "index": "company", "id": "@_id", "label": "@action", "properties": { "time": "@time", "action": "@action", }, "dynamicProperties": true, "outVertex": { "ref": true, "id": "@userId", "label": "user" }, "inVertex": { "ref": true, "id": "@documentId", "label": "document" } } ] } But when I test by following code:

BaseConfiguration conf = new BaseConfiguration(); conf.addProperty("providers","/company/tinkerpop-server/json"); UniGraph graph=UniGraph.open(conf); GraphTraversalSource g=graph.traversal();
System.out.println(g.E().count().toList());

It shows me the count result is 10000 rather than 3170000,I hope to understand what’s strange thing going on?

seanbarzilay commented 6 years ago

This is a known issue Scroll api I'm hoping finish working on it in the next few days. @baozengkai

baozengkai commented 6 years ago

@seanbarzilay Understand, looking forward to better unipop

sorryya commented 6 years ago

From a performance perspective, is it better to use Count api in a count step strategy, rather than Scroll api? @seanbarzilay

seanbarzilay commented 6 years ago

I am aware of that but currently I don't have any Reducing step strategies. #46

sorryya commented 6 years ago

I see. So is there any definite time plan for the Reducing step strategies ? I'm really looking forward to it.

seanbarzilay commented 6 years ago

I'm hoping I can get it done by the next release.