francoispqt / gojay

high performance JSON encoder/decoder with stream API for Golang
MIT License
2.11k stars 113 forks source link

Add functions for stream encoding/decoding of big strings #133

Open xakep666 opened 4 years ago

xakep666 commented 4 years ago

This may be useful i.e. when your service should download file and send it inside json to other service without storing whole file in memory. In my case service downloads a file from S3-like service and makes request to our email provider to send it as attachment (base64 encoded). Currently I'm using this snippet:

func (a *Attach) MarshalJSONObject(e *gojay.Encoder) {
    e.StringKey("name", a.Name)
    e.StringKeyOmitEmpty("encoding", a.Encoding)

    // attach content handling
    if a.Content != nil {
        e.AppendByte(',')                                 // begin new field
        e.AppendBytes([]byte(`"content"`))                // appends "content" key
        e.AppendByte(':')                                 // delimiter between key and value
        e.AppendByte('"')                                 // string begins
        _, err := io.Copy((*encoderWriter)(e), a.Content) // big string (unsafe, no escaping)
        if err != nil {
            panic(err) // caught by caller
        }
        e.AppendByte('"') // string ends
    }
}

My changes allows to shorten it to this:

func (a *Attach) MarshalJSONObject(e *gojay.Encoder) {
    e.StringKey("name", a.Name)
    e.StringKeyOmitEmpty("encoding", a.Encoding)
        e.ReaderToBase64Key("content", a.Content, base64.StdEncoding)
}

This PR adds similar methods for decoding (may be useful i.e. when you get response where file goes as base64-encoded string and this file should be uploaded to somewhere).

Side changes: