search

AntonTerekhov / OrientDB-PHP

Binary protocol for OrientDB for PHP applications (Beta)
http://code.google.com/p/orient/wiki/NetworkBinaryProtocol
BSD 3-Clause "New" or "Revised" License
108 stars 23 forks source link

Memory allocation exhausted when using gremlin. #22

Closed F21 closed 11 years ago

F21 commented 11 years ago

I am trying to run a simple gremlin command:

<?php 

require 'OrientDB/OrientDB.php';

$db = new OrientDB('localhost', 2424);

$db->DBOpen('test','admin', 'admin');

$db->setDebug(true);

var_dump($db->selectGremlin("result = []; result << [test: 'test', testinteger: 1];return result;"
));

This is the result:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 1852255812 bytes) in /www/orientdb-binary/OrientDB/OrientDBSocket.php on line 75

This is the debug value:

0 : 29 00 00 00 66 73 00 00 00 95 00 00 00 3a 63 6f [)...fs.......:co] 10 : 6d 2e 6f 72 69 65 6e 74 65 63 68 6e 6f 6c 6f 67 [m.orientechnolog] 20 : 69 65 73 2e 6f 72 69 65 6e 74 2e 67 72 61 70 68 [ies.orient.graph] 30 : 2e 67 72 65 6d 6c 69 6e 2e 4f 43 6f 6d 6d 61 6e [.gremlin.OComman] 40 : 64 47 72 65 6d 6c 69 6e 00 00 00 44 72 65 73 75 [dGremlin...Dresu] 50 : 6c 74 20 3d 20 5b 5d 3b 20 72 65 73 75 6c 74 20 [lt = []; result ] 60 : 3c 3c 20 5b 74 65 73 74 3a 20 27 74 65 73 74 27 [<< [test: 'test'] 70 : 2c 20 74 65 73 74 69 6e 74 65 67 65 72 3a 20 31 [, testinteger: 1] 80 : 5d 3b 72 65 74 75 72 6e 20 72 65 73 75 6c 74 3b [];return result;] 90 : ff ff ff ff 00 00 00 03 2a 3a 30 00 00 00 00 [........*:0....] >request_status 0 : 00 [.] >TransactionID 0 : 00 00 00 66 [...f] >status 0 : 6c [l] >records_count 0 : 00 00 00 01 [....] >record_marker 0 : 01 00 [..] >record_type 0 : 00 [.] >record_clusterID 0 : 00 66 [.f] >record_position 0 : 01 00 00 00 [....] 0 : 1c 6a 61 76 [.jav] >record_version 0 : 61 2e 6c 61 [a.la] >record_content 0 : 6e 67 2e 43 [ng.C]

I would like to help fix this but I am not sure where to start.

Previously I was using the REST protocol, but there were too many problems with it and it does not look like the issues will be fixed soon. That's why I have decided to use the binary protocol instead.

The binary protocol works pretty well, so I think if we can fix just a few issues with the library, it could be the defacto PHP library for talking with OrientDB :)

AntonTerekhov commented 11 years ago

Hi!

Seems error is happening on OrientDB side.

>record_type      0 : 00 [.] 
>record_clusterID 0 : 00 66 [.f] 
>record_position  0 : 01 00 00 00 [....] 
                  0 : 1c 6a 61 76 [.jav] 
>record_version   0 : 61 2e 6c 61 [a.la] 
>record_content   0 : 6e 67 2e 43 [ng.C]

01 in record_position looks like "exception here", next 00 00 00 1c is exception java class string length and 6a 61 76 61 2e 6c 61 6e 67 2e 43 is java exception class name text "java.lang.C"...

I can't do much here, cause binary protocol don't say anything about possibility of exception text here. Standard place for exception must be here: request_status 0 : must be 01 and next must be exception class name length.

F21 commented 11 years ago

Ah, that's a bit strange, because it works fine in the gremlin console!

I will file an issue with OrientDB and link it here :)

Filed: #1335 with OrientDB :)

F21 commented 11 years ago

@AntonTerekhov:

I just did more investigation. This is in the log on the server:

2013-02-06 09:08:48:024 FINE /127.0.0.1:43736 - Read 150 bytes: \00\00\00:com.orientechnologies.orient.graph.gremlin.OCommandGremlin\00\00\00Eresult = []; result << [test: 'test', testinteger: 1]; return result;����\00\00\00*:0\00\00\00\00 [OChannelBinaryServer]

For some reason, when I open the log file, it says that some characters could not be decoded properly and points to this line.

Could you check if the request is encoded properly?

AntonTerekhov commented 11 years ago

This doen't says anything to me. But in full log at referenced issue I can clearly see Java exception.

Two ways here: 1. this is really Orient DB error. 2. Driver send this gremlin command wrong (gremlin remains experimental). You can send same gremlin via client and compare bytes within logs.

F21 commented 11 years ago

Good idea, I didn't think about sending it using the client! I will try that and post back :)

F21 commented 11 years ago

Ok, so I just tried doing that. Unfortunately, there seems to be a problem with the OrientDB console in sending gremlin queries.

I am running it like this:

orientdb> gremlin 'result=[]; result << [test: "test", testinteger: 1]; return result'

Error: com.orientechnologies.orient.core.exception.OCommandExecutionException: Error on execution of the GREMLIN script
Error: com.orientechnologies.orient.core.exception.OStorageException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 85: expecting ''', found '<EOF>' @ line 85, column 11.
   'result=[]
             ^

1 error

Error: com.orientechnologies.orient.core.exception.OStorageException: startup failed:
Script1.groovy: 85: expecting ''', found '<EOF>' @ line 85, column 11.
   'result=[]
             ^

1 error
!Unrecognized command: 'result << [test: "test", testinteger: 1]'
!Unrecognized command: 'return result''

And this is in the log file:

2013-02-07 10:36:34:417 FINE /127.0.0.1:49558 - Read byte: 41 [OChannelBinaryServer]
2013-02-07 10:36:34:418 FINE /127.0.0.1:49558 - Reading int (4 bytes)... [OChannelBinaryServer]
2013-02-07 10:36:34:418 FINE /127.0.0.1:49558 - Read int: 2 [OChannelBinaryServer]
2013-02-07 10:36:34:418 FINE /127.0.0.1:49558 - Reading byte (1 byte)... [OChannelBinaryServer]
2013-02-07 10:36:34:418 FINE /127.0.0.1:49558 - Read byte: 115 [OChannelBinaryServer]
2013-02-07 10:36:34:419 FINE /127.0.0.1:49558 - Reading chunk of bytes. Reading chunk length as int (4 bytes)... [OChannelBinaryServer]
2013-02-07 10:36:34:419 FINE /127.0.0.1:49558 - Read chunk lenght: 78 [OChannelBinaryServer]
2013-02-07 10:36:34:419 FINE /127.0.0.1:49558 - Reading 78 bytes... [OChannelBinaryServer]
2013-02-07 10:36:34:419 FINE /127.0.0.1:49558 - Read 78 bytes: \00\00\00:com.orientechnologies.orient.graph.gremlin.OCommandGremlin\00\00\00
'result=[]\00\00 [OChannelBinaryServer]
2013-02-07 10:36:35:606 FINE /127.0.0.1:49558 - Flush [OChannelBinaryServer]
2013-02-07 10:36:35:607 FINE /127.0.0.1:49558 - Writing byte (1 byte): 1 [OChannelBinaryServer]
2013-02-07 10:36:35:607 FINE /127.0.0.1:49558 - Writing int (4 bytes): 2 [OChannelBinaryServer]
2013-02-07 10:36:35:607 FINE /127.0.0.1:49558 - Writing byte (1 byte): 1 [OChannelBinaryServer]
2013-02-07 10:36:35:608 FINE /127.0.0.1:49558 - Writing string (4+70=74 bytes): com.orientechnologies.orient.core.exception.OCommandExecutionException [OChannelBinaryServer]
2013-02-07 10:36:35:608 FINE /127.0.0.1:49558 - Writing string (4+40=44 bytes): Error on execution of the GREMLIN script [OChannelBinaryServer]
2013-02-07 10:36:35:608 FINE /127.0.0.1:49558 - Writing byte (1 byte): 1 [OChannelBinaryServer]
2013-02-07 10:36:35:608 FINE /127.0.0.1:49558 - Writing string (4+28=32 bytes): javax.script.ScriptException [OChannelBinaryServer]
2013-02-07 10:36:35:608 FINE /127.0.0.1:49558 - Writing string (4+189=193 bytes): org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 85: expecting ''', found '<EOF>' @ line 85, column 11.
   'result=[]
             ^

1 error
 [OChannelBinaryServer]
2013-02-07 10:36:35:609 FINE /127.0.0.1:49558 - Writing byte (1 byte): 1 [OChannelBinaryServer]
2013-02-07 10:36:35:609 FINE /127.0.0.1:49558 - Writing string (4+62=66 bytes): org.codehaus.groovy.control.MultipleCompilationErrorsException [OChannelBinaryServer]
2013-02-07 10:36:35:615 FINE /127.0.0.1:49558 - Writing string (4+125=129 bytes): startup failed:
Script1.groovy: 85: expecting ''', found '<EOF>' @ line 85, column 11.
   'result=[]
             ^

1 error
 [OChannelBinaryServer]
2013-02-07 10:36:35:615 FINE /127.0.0.1:49558 - Writing byte (1 byte): 0 [OChannelBinaryServer]
2013-02-07 10:36:35:615 FINE /127.0.0.1:49558 - Flush [OChannelBinaryServer]Sent run-time exception to the client /127.0.0.1:49558: com.orientechnologies.orient.core.exception.OCommandExecutionException: Error on execution of the GREMLIN script
Error on execution of the GREMLIN script
-> com.orientechnologies.orient.graph.gremlin.OGremlinHelper.execute(OGremlinHelper.java:201)
-> com.orientechnologies.orient.graph.gremlin.OCommandGremlinExecutor.execute(OCommandGremlinExecutor.java:51)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:95)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.command(OStorageEmbedded.java:84)
-> com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:60)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1130)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:295)
-> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:120)
-> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:43)
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 85: expecting ''', found '<EOF>' @ line 85, column 11.
   'result=[]
             ^

1 error

-> com.orientechnologies.orient.graph.gremlin.OGremlinHelper.execute(OGremlinHelper.java:201)
-> com.orientechnologies.orient.graph.gremlin.OCommandGremlinExecutor.execute(OCommandGremlinExecutor.java:51)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:95)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.command(OStorageEmbedded.java:84)
-> com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:60)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1130)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:295)
-> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:120)
-> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:43)
startup failed:
Script1.groovy: 85: expecting ''', found '<EOF>' @ line 85, column 11.
   'result=[]
             ^

1 error

-> com.orientechnologies.orient.graph.gremlin.OGremlinHelper.execute(OGremlinHelper.java:201)
-> com.orientechnologies.orient.graph.gremlin.OCommandGremlinExecutor.execute(OCommandGremlinExecutor.java:51)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:95)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.command(OStorageEmbedded.java:84)
-> com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:60)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1130)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:295)
-> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:120)
-> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:43)
2013-02-07 10:36:35:636 FINE /127.0.0.1:49558 - Reading byte (1 byte)... [OChannelBinaryServer]

If I do this, I also get an unrecognized command:

orientdb> gremlin result=[]; result << [test: "test", testinteger: 1]; return result;

[]

Script executed in 0.110000 sec(s).!Unrecognized command: 'result << [test: "test", testinteger: 1]'
!Unrecognized command: 'return result'

And the log:

2013-02-07 10:42:04:733 FINE /127.0.0.1:49557 - Read byte: 41 [OChannelBinaryServer]
2013-02-07 10:42:04:734 FINE /127.0.0.1:49557 - Reading int (4 bytes)... [OChannelBinaryServer]
2013-02-07 10:42:04:734 FINE /127.0.0.1:49557 - Read int: 2 [OChannelBinaryServer]
2013-02-07 10:42:04:734 FINE /127.0.0.1:49557 - Reading byte (1 byte)... [OChannelBinaryServer]
2013-02-07 10:42:04:734 FINE /127.0.0.1:49557 - Read byte: 115 [OChannelBinaryServer]
2013-02-07 10:42:04:735 FINE /127.0.0.1:49557 - Reading chunk of bytes. Reading chunk length as int (4 bytes)... [OChannelBinaryServer]
2013-02-07 10:42:04:735 FINE /127.0.0.1:49557 - Read chunk lenght: 77 [OChannelBinaryServer]
2013-02-07 10:42:04:735 FINE /127.0.0.1:49557 - Reading 77 bytes... [OChannelBinaryServer]
2013-02-07 10:42:04:736 FINE /127.0.0.1:49557 - Read 77 bytes: \00\00\00:com.orientechnologies.orient.graph.gremlin.OCommandGremlin\00\00\00    result=[]\00\00 [OChannelBinaryServer]
2013-02-07 10:42:04:866 FINE /127.0.0.1:49557 - Writing byte (1 byte): 0 [OChannelBinaryServer]
2013-02-07 10:42:04:866 FINE /127.0.0.1:49557 - Writing int (4 bytes): 2 [OChannelBinaryServer]
2013-02-07 10:42:04:866 FINE /127.0.0.1:49557 - Writing byte (1 byte): 108 [OChannelBinaryServer]
2013-02-07 10:42:04:866 FINE /127.0.0.1:49557 - Writing int (4 bytes): 0 [OChannelBinaryServer]
2013-02-07 10:42:04:866 FINE /127.0.0.1:49557 - Flush [OChannelBinaryServer]
2013-02-07 10:42:04:870 FINE /127.0.0.1:49557 - Reading byte (1 byte)... [OChannelBinaryServer]

Unfortunately, if I try doing it in the gremlin console, I don't get anything in the logs as it uses blueprints directly and not the binary protocol :(

lvca commented 11 years ago

Have you tried to just remove the last part "; return result;" ? Try executing this:

gremlin result=[]; result << [test: "test", testinteger: 1]

F21 commented 11 years ago

Hi Luca,

If I tried doing that using the PHP library, same error:
0 : 29 00 00 00 03 73 00 00 00 87 00 00 00 3a 63 6f [)....s.......:co] 10 : 6d 2e 6f 72 69 65 6e 74 65 63 68 6e 6f 6c 6f 67 [m.orientechnolog] 20 : 69 65 73 2e 6f 72 69 65 6e 74 2e 67 72 61 70 68 [ies.orient.graph] 30 : 2e 67 72 65 6d 6c 69 6e 2e 4f 43 6f 6d 6d 61 6e [.gremlin.OComman] 40 : 64 47 72 65 6d 6c 69 6e 00 00 00 36 72 65 73 75 [dGremlin...6resu] 50 : 6c 74 20 3d 20 5b 5d 3b 20 72 65 73 75 6c 74 20 [lt = []; result ] 60 : 3c 3c 20 5b 74 65 73 74 3a 20 27 74 65 73 74 27 [<< [test: 'test'] 70 : 2c 20 74 65 73 74 69 6e 74 65 67 65 72 3a 20 31 [, testinteger: 1] 80 : 5d 3b ff ff ff ff 00 00 00 03 2a 3a 30 00 00 00 [];........*:0...] 90 : 00 [.] >request_status 0 : [] 0 : 00 [.] >TransactionID 0 : 00 00 00 03 [....] >status 0 : 6c [l] >records_count 0 : 00 00 00 01 [....] >record_marker 0 : 01 00 [..] >record_type 0 : 00 [.] >record_clusterID 0 : 00 03 [..] >record_position 0 : 01 00 00 00 [....] 0 : 1c 6a 61 76 [.jav] >record_version 0 : 61 2e 6c 61 [a.la] >record_content 0 : 6e 67 2e 43 [ng.C]

( ! ) Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 1852255812 bytes) in /www/orientdb-binary/OrientDB/OrientDBSocket.php on line 75

If I try doing that in the console, also the same problem:

orientdb> gremlin result=[]; result << [test: "test", testinteger: 1]

[]

Script executed in 0.131000 sec(s).!Unrecognized command: 'result << [test: "test", testinteger: 1]'
AntonTerekhov commented 11 years ago

I see that this issue remains unclear.

Can you check bytes from OrientDB-PHP and native client with some packet inspector, like Wireshark? Its easy to find sequence of bytes in WIreshark as with debug mode driver echoes bytes to stdout.