Graph Editor - canvas is not populated

[santo-it]

Hi,

Under certain conditions writing a query will not produce any graph I saw this many times now, and it has not been introduced by recent versions

I'm still unsure if this is related to studio or distributed. Opening this issue anyway

Easy way to reproduce for me is the following:

• start a 3 nodes cluster
• create a db in node1. As result of DB creation Studio will connect to the database using the root user (or the one that has been used to create the database) - (will open a new issue on this)
• now a script loads some vertices and edges in a concurrent way on node1, node2 and node3
• when the script finishes, I try to select * from E from node 1 (db connection is already open, with root user) --> no results
• if I try to execute the same query from node2 and node3, it works
• note that from node1 I do see the vertices and edges in the schema panel - just the graph do not show them
• if from node1 I try to disconnect and reconnect it works

Thanks,

[SindhujaRG]

Hi,

Any update on this issue? I have been facing the same issue.

Thanks in advance.

[SLENGLET]

Hi ,

I've also the same problem : ok in schema panel , but nothing in the Graph with same query

[wolf4ood]

Hi @SLENGLET

which query are you running?

[SLENGLET]

Just a simple "traverse" like this :

traverse * from #37:1424624

I see the result in the schema panel , but nothing Graph panel .

I also see many logs like this : "Magic number verification failed for page ..."

[wolf4ood]

@slenglet

which version are you using?

[SLENGLET]

By the way , let me describe the situation . My DB has 120M vertices et 60M edges . I've used the "oetl.sh" to do this . During an import , i ve decided to stop the process with a "kill -9" because the "row/s import " was to slow. After this operation, and stop/start the server, graph editor was down ( all queries)

[SLENGLET]

version: orientdb-community-importers-2.2.30

[wolf4ood]

@slenglet

so i guess it's not Graph Editor problem.

Can you share the logs of the Server?. Probably there is something related to the kill -9

Thanks

[SLENGLET]

Sorry i can't share logs because each time i stop/start the server , logs are lost ( no rotation logs, i know it's not a good practice , but that's the situation )

[SLENGLET]

i finally found logs from start server just after the "kill -9 oetl pid"

2018-03-28 13:34:02:080 INFO Loading configuration from: /logiciels/orientdb/orientdb-community-importers-2.2.30/config/orientdb-server-config.xml... [OServerConfigurationLoaderXml] 2018-03-28 13:34:02:499 INFO OrientDB Server v2.2.30 (build 43d850df23c0fd1edca7279761c2238fdc6cc576, branch 2.2.x) is starting up... [OServer] 2018-03-28 13:34:02:507 INFO Databases directory: /appli/projects/DCO-EDFCO-V2/orientdb/databases [OServer] 2018-03-28 13:34:02:566 INFO Auto configuration of disk cache size. [OMemoryAndLocalPaginatedEnginesInitializer] 2018-03-28 13:34:02:695 INFO 33568837632 B/32013 MB/31 GB of physical memory were detected on machine [ONative] 2018-03-28 13:34:02:704 INFO Soft memory limit for this process is set to -1 B/-1 MB/-1 GB [ONative] 2018-03-28 13:34:02:705 INFO Hard memory limit for this process is set to -1 B/-1 MB/-1 GB [ONative] 2018-03-28 13:34:02:705 INFO Path to 'memory' cgroup is '/' [ONative] 2018-03-28 13:34:02:709 INFO Mounting path for memory cgroup controller is '/sys/fs/cgroup/memory' [ONative] 2018-03-28 13:34:02:709 INFO cgroup soft memory limit is 9223372036854771712 B/8796093022207 MB/8589934591 GB [ONative] 2018-03-28 13:34:02:710 INFO cgroup hard memory limit is 9223372036854771712 B/8796093022207 MB/8589934591 GB [ONative] 2018-03-28 13:34:02:711 INFO Detected memory limit for current process is 33568837632 B/32013 MB/31 GB [ONative] 2018-03-28 13:34:02:713 INFO OrientDB auto-config DISKCACHE=28,002MB (heap=1,963MB direct=524,288MB os=32,013MB) [OMemoryAndLocalPaginatedEnginesInitializer] 2018-03-28 13:34:02:936 INFO Listening binary connections on 0.0.0.0:2424 (protocol v.36, socket=default) [OServerNetworkListener] 2018-03-28 13:34:02:940 INFO Listening http connections on 0.0.0.0:2480 (protocol v.10, socket=default) [OServerNetworkListener] 2018-03-28 13:34:02:955 INFO Installing dynamic plugin 'orientdb-studio-2.2.30.zip'... [OServerPluginManager] 2018-03-28 13:34:02:966 INFO Installing dynamic plugin 'orientdb-etl-2.2.30.jar'... [OServerPluginManager] 2018-03-28 13:34:02:972 INFO Installing dynamic plugin 'orientdb-teleporter-2.2.30.jar'... [OServerPluginManager] 2018-03-28 13:34:02:988 INFO Installing dynamic plugin 'orientdb-neo4j-importer-plugin-2.2.30-dist.jar'... [OServerPluginManager] 2018-03-28 13:34:02:994 INFO ODefaultPasswordAuthenticator is active [ODefaultPasswordAuthenticator] 2018-03-28 13:34:02:995 INFO OServerConfigAuthenticator is active [OServerConfigAuthenticator] 2018-03-28 13:34:02:996 INFO OSystemUserAuthenticator is active [OSystemUserAuthenticator] 2018-03-28 13:34:03:023 INFO Installed GREMLIN language v.2.6.0 - graph.pool.max=1000 [OGraphServerHandler] 2018-03-28 13:34:03:023 INFO [OVariableParser.resolveVariables] Error on resolving property: distributed [orientechnologies] 2018-03-28 13:34:03:028 WARNI Authenticated clients can execute any kind of code into the server by using the following allowed languages: [sql] [OServerSideScriptInterpreter] 2018-03-28 13:34:03:032 INFO OrientDB Studio available at http://10.119.5.57:2480/studio/index.html [OServer] 2018-03-28 13:34:03:032 INFO OrientDB Server is active v2.2.30 (build 43d850df23c0fd1edca7279761c2238fdc6cc576, branch 2.2.x). [OServer] 2018-03-28 13:34:11:482 WARNI {db=OSystem} Storage 'OSystem' was not closed properly. Will try to recover from write ahead log... [OLocalPaginatedStorage] 2018-03-28 13:34:11:483 INFO {db=OSystem} Looking for last checkpoint... [OLocalPaginatedStorage] 2018-03-28 13:34:11:487 INFO {db=OSystem} Found FUZZY checkpoint. [OLocalPaginatedStorage] 2018-03-28 13:34:11:487 INFO {db=OSystem} Data restore procedure from FUZZY checkpoint is started. [OLocalPaginatedStorage] 2018-03-28 13:34:11:490 WARNI {db=OSystem} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointEndRecord{lsn=LSN{segment=0, position=1007377}} will be skipped during data restore [OLocalPaginatedStorage] 2018-03-28 13:34:11:490 INFO {db=OSystem} 1 operations were processed, current LSN is LSN{segment=0, position=1007377} last LSN is LSN{segment=0, position=1007424} [OLocalPaginatedStorage] 2018-03-28 13:34:11:491 WARNI {db=OSystem} Record OFuzzyCheckpointStartRecord{lsn=LSN{segment=0, position=1007384}} com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointStartRecord{lsn=null, previousCheckpoint=LSN{segment=0, position=1007337}} will be skipped during data restore [OLocalPaginatedStorage] 2018-03-28 13:34:11:492 WARNI {db=OSystem} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointEndRecord{lsn=LSN{segment=0, position=1007424}} will be skipped during data restore [OLocalPaginatedStorage] 2018-03-28 13:34:11:507 INFO {db=OSystem} Storage data recover was completed [OLocalPaginatedStorage] 2018-03-28 13:34:11:888 INFO {db=OSystem} Storage 'plocal:/appli/projects/DCO-EDFCO-V2/orientdb/databases/OSystem' is opened under OrientDB distribution : 2.2.30 (build 43d850df23c0fd1edca7279761c2238fdc6cc576, branch 2.2.x) [OLocalPaginatedStorage] 2018-03-28 13:34:52:276 WARNI {db=GraphMOE2} Storage 'GraphMOE2' was not closed properly. Will try to recover from write ahead log... [OLocalPaginatedStorage] 2018-03-28 13:34:52:276 INFO {db=GraphMOE2} Looking for last checkpoint... [OLocalPaginatedStorage] 2018-03-28 13:34:52:277 INFO {db=GraphMOE2} Found FUZZY checkpoint. [OLocalPaginatedStorage] 2018-03-28 13:34:52:278 INFO {db=GraphMOE2} Data restore procedure from FUZZY checkpoint is started. [OLocalPaginatedStorage] 2018-03-28 13:34:52:278 WARNI {db=GraphMOE2} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointEndRecord{lsn=LSN{segment=407, position=98303080}} will be skipped during data restore [OLocalPaginatedStorage] 2018-03-28 13:34:52:278 INFO {db=GraphMOE2} 1 operations were processed, current LSN is LSN{segment=407, position=98303080} last LSN is LSN{segment=407, position=98303127} [OLocalPaginatedStorage] 2018-03-28 13:34:52:279 WARNI {db=GraphMOE2} Record OFuzzyCheckpointStartRecord{lsn=LSN{segment=407, position=98303087}} com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointStartRecord{lsn=null, previousCheckpoint=LSN{segment=407, position=98303040}} will be skipped during data restore [OLocalPaginatedStorage] 2018-03-28 13:34:52:279 WARNI {db=GraphMOE2} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OFuzzyCheckpointEndRecord{lsn=LSN{segment=407, position=98303127}} will be skipped during data restore [OLocalPaginatedStorage] 2018-03-28 13:34:52:312 INFO {db=GraphMOE2} Storage data recover was completed [OLocalPaginatedStorage] 2018-03-28 13:34:52:364 SEVER {db=GraphMOE2} Magic number verification failed for page '4' of 'IsLinkedToInterLoc.in_out.hit'. [OWOWCache] 2018-03-28 13:34:52:462 INFO {db=GraphMOE2} Previous maximum cache size was 400982 current maximum cache size is 380827. Cache state for storage /appli/projects/DCO-EDFCO-V2/orientdb/databases/GraphMOE2 will not be restored. [O2QCache] 2018-03-28 13:34:52:463 INFO {db=GraphMOE2} Storage 'plocal:/appli/projects/DCO-EDFCO-V2/orientdb/databases/GraphMOE2' is opened under OrientDB distribution : 2.2.30 (build 43d850df23c0fd1edca7279761c2238fdc6cc576, branch 2.2.x) [OLocalPaginatedStorage] 2018-03-28 13:35:55:880 SEVER {db=GraphMOE2} Magic number verification failed for page '19153' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:880 SEVER {db=GraphMOE2} Magic number verification failed for page '19154' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:881 SEVER {db=GraphMOE2} Magic number verification failed for page '19155' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:881 SEVER {db=GraphMOE2} Magic number verification failed for page '19156' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:882 SEVER {db=GraphMOE2} Magic number verification failed for page '19157' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:882 SEVER {db=GraphMOE2} Magic number verification failed for page '19158' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:883 SEVER {db=GraphMOE2} Magic number verification failed for page '19159' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:883 SEVER {db=GraphMOE2} Magic number verification failed for page '19160' of 'activite.pcl'. [OWOWCache] 2018-03-28 13:35:55:883 SEVER {db=GraphMOE2} Magic number verification failed for page '19161' of 'activite.pcl'. [OWOWCache] .......

[SLENGLET]

I'v decided to do a new import in a new DB . Unfortunaly, after hours without troubles, import is very very very slow

2018-03-29 11:19:15:362 INFO + extracted 20,536 rows (4 rows/sec) - 20,536 rows -> loaded 20,034 vertices (4 vertices/sec) Total time: 1862979ms [0 warnings, 0 errors] [OETLProcessor] 2018-03-29 11:19:16:642 INFO + extracted 20,541 rows (3 rows/sec) - 20,541 rows -> loaded 20,039 vertices (3 vertices/sec) Total time: 1864259ms [0 warnings, 0 errors] [OETLProcessor] 2018-03-29 11:19:17:643 INFO + extracted 20,544 rows (2 rows/sec) - 20,544 rows -> loaded 20,042 vertices (2 vertices/sec) Total time: 1865260ms [0 warnings, 0 errors] [OETLProcessor] 2018-03-29 11:19:18:643 INFO + extracted 20,550 rows (6 rows/sec) - 20,550 rows -> loaded 20,048 vertices (6 vertices/sec) Total time: 1866260ms [0 warnings, 0 errors] [OETLProcessor] 2018-03-29 11:19:19:758 INFO + extracted 20,553 rows (2 rows/sec) - 20,553 rows -> loaded 20,051 vertices (2 vertices/sec) Total time: 1867375ms [0 warnings, 0 errors] [OETLProcessor]^C

--> 8M have to be imported !

My question is simple : if i want to stop oetl.sh process properly ( so no "kill -9 pid" ) , what do i have to do ?

Regards

[wolf4ood]

Hi @slenglet

can you share the etl configuration?

[SLENGLET]

Hi

{ "source": { "file": { "path": "/appli/projects/DCO-EDFCO-V2/orientdb/init/OrientDBObjectPointEnergie/vertex.csv" } }, "extractor": { "csv": {} }, "transformers": [ {"vertex": { "class": "PE" ,"skipDuplicates": true} } ], "loader": { "orientdb": { "dbURL": "plocal:/logiciels/orientdb/orientdb-community-importers-2.2.30/databases/GraphMOE3", "dbType": "graph", "tx": false, "wal": false, "batchCommit": 10000, "classes": [ {"name": "PE", "extends": "V"} ], "indexes": [ {"class":"PE", "fields":["IdPE:String"], "type":"UNIQUE_HASH_INDEX" } ] } } }

[wolf4ood]

I'd try to use kill -15

[wolf4ood]

hi @slenglet

which is the initial speed of the ETL?

are you running it with ./etl.sh <json>?

Thanks

[SLENGLET]

Around 10 000 rows/s for vertices and 1000 rows/s for edges when all is ok

Yes i use ../orientdb-community-importers-2.2.30/bin/oetl.sh json

[wolf4ood]

Is it the slowdown on edges or on vertices?

[SLENGLET]

Edges !

By the way i sent you a wrong file .

Here is the correct file { "source": { "file": { "path": "/appli/projects/DCO-EDFCO-V2/orientdb/init/OrientDBJoinAdressePointEnergie/edges.csv" } }, "extractor": { "csv": {} }, "transformers": [ { "merge": { "joinFieldName": "IdAdresse", "lookup": "Adresse.IdAdresse" } }, { "vertex": { "class": "Adresse","skipDuplicates": true } }, { "edge": { "class": "IsLinkedToPE", "joinFieldName": "IdPE", "lookup": "PE.IdPE", "skipDuplicates": true, "unresolvedLinkAction": "CREATE" } } ], "loader": { "orientdb": { "dbURL": "plocal:/logiciels/orientdb/orientdb-community-importers-2.2.30/databases/GraphMOE3", "dbType": "graph", "tx": false, "wal": false, "batchCommit": 10000, "classes": [ {"name": "Adresse", "extends": "V"}, {"name": "IsLinkedToPE", "extends": "E"} ], "indexes": [ {"class":"IsLinkedToPE", "fields":["in:LINK","out:LINK"], "type":"UNIQUE_HASH_INDEX" } ] } } }

[SLENGLET]

I maybe found the reason : vertices "PointEnergie" are not yet created .

could it be the reason ?

[wolf4ood]

hi @slenglet

could be. Btw you should also increate i think the memory settings in oetl.sh

JAVA_OPTS=-Xmx512m

to

JAVA_OPTS=-Xmx1G

It could be a slowdown due GC activity

[SLENGLET]

Thx Enrico, i try those changes . i will keep you updated

[SLENGLET]

Hi @maggiolo00

Many thanks for your help . --> Several imports via crontab this night and all seems ok ( rows/s is normal again )

[v1nsai]

I'm having a similar issue. I can see everything I loaded in the graph in the Browse tab using a simple select from v, but the same thing in the Graph Editor shows nothing.

Other databases on the server (that I did not create) have no issue showing results in the Graph Editor, so I feel like I may have misconfigured something, but I don't see anything in options that seems relevant.

I haven't messed with any backend options, I simply created a new database, loaded SQL schema and data, then queried with no result in Graph Editor but the same query gets results in Browse.

EDIT: This turned out to be an issue on my end. Once they reinstalled the server it started displaying again.