Passing large string back to mruby

[errordeveloper]

I'm trying to pass long strings using m.LoadString("..."), and hit error like:

line 1:552695: string too long (truncated)

I've looked at the code, and found a seemingly non-configurable constant:

include/mruby/compile.h:#define MRB_PARSER_BUF_SIZE 1024

I guess changing MRB_PARSER_BUF_SIZE to a very big number would work, but it's still gonna be a fixed number and I can imagine this is a statically allocated buffer and it's really dull.

The question is: what's the better way to return a long string as a value without resizing parser buffer? Is there a chunked parser mode of sorts?

[errordeveloper]

One approach I can think of is to split my long string into chunks of length MRB_PARSER_BUF_SIZE, and pop those into a Ruby array to join later. What would be other options?

[erikh]

I'll need to look into why, but this is probably not a go-mruby bug, but a mruby issue. I'll see if I can repro with a C program I can send to them.

[errordeveloper]

@erikh I didn't mean this as a bug report, it's more of a question. I'd accept that this as a limitation in mruby.

[erikh]

Ah. Well, I will probably still look into it. :D

On Mon, Dec 19, 2016 at 5:28 AM, Ilya Dmitrichenko <notifications@github.com

wrote:

@erikh https://github.com/erikh I didn't mean this as a bug report, it's more of a question. I'd accept that this is a limitation in mruby.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mitchellh/go-mruby/issues/46#issuecomment-267963653, or mute the thread https://github.com/notifications/unsubscribe-auth/AABJ6xYoDtiwSa12C0jWBOk3-8n7oRzBks5rJoZggaJpZM4LQrrB .

[errordeveloper]

So first naive version I tried was this:

func marshalToJSON(obj interface{}, m *mruby.Mrb) (mruby.Value, mruby.Value) {
    data, err := json.MarshalIndent(obj, "", "  ")
    if err != nil {
        return nil, createException(m, err.Error())
    }

    value, err := m.LoadString(fmt.Sprintf("%q", data))
    if err != nil {
        return nil, createException(m, err.Error())
    }
    return value, nil
}

And it didn't work.

Now I've made it into this:

func bytesToStringSlice(data []byte, chunkSize int) (chunks []string) {
    stringData := string(data)
    for {
        if len(stringData) >= chunkSize {
            chunks, stringData = append(chunks, stringData[0:chunkSize]), stringData[chunkSize:]
        } else {
            chunks, stringData = append(chunks, stringData[0:len(stringData)]), ""
            break
        }
    }
    return chunks
}

func marshalToJSON(obj interface{}, m *mruby.Mrb) (mruby.Value, mruby.Value) {
    data, err := json.MarshalIndent(obj, "", "  ")
    if err != nil {
        return nil, createException(m, err.Error())
    }

    value, err := m.LoadString(fmt.Sprintf("@tmp = []"))
    if err != nil {
        return nil, createException(m, err.Error())
    }
    for _, chunk := range bytesToStringSlice(data, 512) {
        _, err := m.LoadString(fmt.Sprintf("@tmp << %q", chunk))
        if err != nil {
            return nil, createException(m, err.Error())
        }
    }

    return value, nil
}

And it works, but I am convinced there should exist a more elegant solution.

[errordeveloper]

Turns out m.StringValue("...") is all I needed! Next time I should have a better look at the docs first, and not read the unit tests...