Vote for New Feature: generate Scan (as specified by sql.Scanner) and driver.Valuer (as specified by driver.Valuer)

[pascallouisperez]

To store protos in a db (as blobs), it would be great to have an extension generating:

• Sacn https://golang.org/pkg/database/sql/#Scanner
• Value https://golang.org/pkg/database/sql/driver/#Valuer

[awalterschulze]

I am not really a sql user, could you please explain how this would work in a little more detail? I am also pretty sure that sql does not support nested structures, but protocol buffers do, can the Scanner or Valuer interfaces handle nested structures?

[chancez]

This would be awesome. @awalterschulze Scanners can handle nested structures, but it does get a bit difficult, you end up calling the Scanner method of the nested type on some bytes. However, even without supporting that, this would be quite valuable. Also, because we know the schema ahead of time, it should be possible to do Scanning without reflection, which would be even better.

[awalterschulze]

Maybe someone could give me an example of what the generated code would look like for an example structure?

[pascallouisperez]

Apologies for being MIA despite my initial ask. Providing examples is on my TODO list, but swamped right now!

[tamird]

@awalterschulze if I understand this correctly, the only feasible implementation of Valuer would be a straight delegation to Marshal, and the only reasonable implementation of Scanner would be a straight delegation to Unmarshal (which would assert that src is a []byte). @pascallouisperez did I understand your intent correctly?

[tamird]

@ecnahc515 you cannot implement Scanner in a way that scans multiple fields into a single structure.

[pascallouisperez]

@tamird Would that support NULL fields?

[tamird]

I don't see why not. The hardest part of all this will be adding tests, since there's no SQL anything in gogoproto right now.

[awalterschulze]

Is this still a thing you want?

[bir]

I would potentially like to revisit this. Although, I am not in need of complex proto message scan/value support for nesting -> but simply the wrappers defined here.

As an example, adding the following to wrappers.pb.go enables the use of a message with a StringValue to be read directly with a SQL call :

func (s *StringValue) Scan(value interface{}) error {
    if value == nil {
        s.Reset()
        return nil
    }
    if bv, err := driver.String.ConvertValue(value); err == nil {
        if v, ok := bv.(string); ok {
            *s = StringValue{v}
            return nil
        }
    }
    return errors.New("failed to scan StringValue")
}

Without this patch you get the following type of errors:

sql: Scan error on column index 1: unsupported Scan, storing driver.Value type string into type *types.StringValue

Right now I have resolved this by forking the repo and manually adding the Scan and Value methods I need. Happy to create a PR, however, I realize that those files are actually generated and adding support for this to the generator would be the potentially proper solution. If you can point me in the right direction I can dig into that.

[awalterschulze]

I would recommend building a plugin and then a vanity binary. That way you can have a decoupled project and still use all the other gogoprotobuf features if you want. Here are some pointers on how to do that https://github.com/gogo/protobuf/issues/304

[travisjeffery]

https://github.com/travisjeffery/proto-go-sql - generator I wrote that'll implement sql.Scanner and driver.Valuer

[awalterschulze]

Nice 😁 could we add your project to the user list in gogoprotobuf?

On Tue, 27 Mar 2018, 20:00 Travis Jeffery, notifications@github.com wrote:

https://github.com/travisjeffery/proto-go-sql - generator I wrote that'll implement sql.Scanner and driver.Valuer

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/gogo/protobuf/issues/108#issuecomment-376618851, or mute the thread https://github.com/notifications/unsubscribe-auth/ABvsLbRONEZx-Nv61G2Rq4AvP6FO8ms2ks5tin5FgaJpZM4GDSqk .