Size limits on queries?

[kushal256]

Seeing some size limitations on reading and writing, wondering if there are any limits that are configurable on client side?

For reading:

{

    for i := 0; i < 2000; i++ {
        res, err := g.Execute(
            fmt.Sprintf("g.addV('label').property('testString','%v')", i),
            nil)
        if err != nil {
            panic(err)
        }
        fmt.Println(res)
    }

    {
        res, err := g.Execute(
            "g.V().values('testString')",
            nil) 
        if err != nil {
            panic(err)
        }
        fmt.Println(res)   //<---- Expecting 2000 rows, only seeing around 60.  Works via java driver, so don't think this is a setting with my gremlin server
    }
}

For writing:

{
    longString := ""
    for i:=0 ;i<3900;i++{ //3800 works, but 3900 doesn't and returns error.  Didn't test this via java driver, can do so if it helps
        longString += "1"
        //fmt.Println("longString = ", longString)
    }
    fmt.Println("len(longString) = ", len(longString))

    szLongInsert := fmt.Sprintf("g.addV('longStringInsert').property('testString','%v')", longString)
    fmt.Println("len(szLongInsert) = ", len(szLongInsert))
    res, err := g.Execute(
        szLongInsert,
        nil)
    if err != nil { 
        panic(err)
    }
    fmt.Println(res)
}

[qasaur]

Hi,

Could you post the error that you are receiving when executing the code you provided?

[kushal256]

"For writing" block of code generates this error:

panic: interface conversion: interface is nil, not string

goroutine 1099 [running]: panic(0x3d7640, 0xc820126040) /usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6 github.com/qasaur/gremgo.marshalResponse(0xc8208e4600, 0x8a, 0x600, 0x3b78e0, 0xc820526bf0, 0x0, 0x0, 0x1f3, 0x0, 0x0) /Users//dev/gowork/src/github.com/qasaur/gremgo/response.go:43 +0x39b github.com/qasaur/gremgo.(_Client).handleResponse(0xc820012480, 0xc8208e4600, 0x8a, 0x600, 0x0, 0x0) /Users//dev/gowork/src/github.com/qasaur/gremgo/response.go:15 +0x56 created by github.com/qasaur/gremgo.(_Client).readWorker /Users//dev/gowork/src/github.com/qasaur/gremgo/connection.go:66 +0x193 exit status 2

And from server side shows this error: [WARN] AbstractGraphSONMessageSerializerV1d0 - Request [PooledUnsafeDirectByteBuf(ridx: 4079, widx: 4079, cap: 4096)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0. [WARN] OpSelectorHandler - Invalid OpProcessor requested [null] org.apache.tinkerpop.gremlin.server.op.OpProcessorException: Invalid OpProcessor requested [null] at org.apache.tinkerpop.gremlin.server.handler.OpSelectorHandler.decode(OpSelectorHandler.java:93) at org.apache.tinkerpop.gremlin.server.handler.OpSelectorHandler.decode(OpSelectorHandler.java:50) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:307) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:293) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:307)

[qasaur]

It seems like the problem was not any size limit but some timing issues with retrieving responses. I think I have fixed the issue now in the latest commit and it seems good on my side, but I'll wait until it has been resolved on your side before closing this issue.

[kushal256]

'For reading' test works now, thanks!

Still seeing an issue with the 'For writing' test

[kushal256]

Additionally, with this latest commit, this command blocks indefinitely and doesn't return anything:

data, err = gremlin.Execute("g.V().drop()", nil) fmt.Println("data ", data) if err != nil { panic(err) }

before it used to block until the command was finished executing

[qasaur]

It looks like the problem with the write query stemmed from the fact that you were trying to insert a very long string (an edge case I did not account for) and that the WebSocket client's read and write buffers were too small for it (4096 bytes) which lead to a malformed query and an error. I doubled the buffer sizes to 8192 bytes in the latest commit and the problem seems to have been fixed.

As for the blocking query bug, this was because the query "g.V().drop()" returns a code of 204 indicating an empty message but completed stream while gremgo only accepted code 200 as a signal for a completed stream, which resulted in the query being blocked since the client was waiting for an additional message with code 200. This bug has also been fixed in the latest commit, so it should hopefully work as expected on your end now.

[kushal256]

Awesome, everything is looking good!

[qasaur]

Great! Closing this issue now.

[rossb83]

Just a comment that it looks like 8192 buffer size is only useful for queries of max size 7999 bytes. Is there a way to dynamically support any query size without continually increasing buffer size?