gogo / protobuf

[Deprecated] Protocol Buffers for Go with Gadgets
Other
5.65k stars 810 forks source link

Support database scanning for wrapper and timestamp Well Known Types #625

Open veqryn opened 4 years ago

veqryn commented 4 years ago

I would like a way to scan/marshal directly from my databases to my protobuf message structs, that way I don't have to maintain and constantly translate between two versions of my data model in my code (the regular golang struct version, and the protobuf + generated message version).

This works fine, until you have timestamps and nullable fields in your database (a pretty typical occurrence in every db i've made or worked with).

To give an example of what I would like to be able to do: Golang database scanning code (using sqlx struct scanning) :

func (db *sqlxDB) CustomerGet(ctx context.Context, id uint64) (*pb.Customer, error) {
    rval := &pb.Customer{}
    err := db.sqlxConn.GetContext(ctx, rval, `
        SELECT id, short_name, created_at
        FROM customer WHERE id = $1`,
        id)
    return rval, err
}

Protobuf definition:

message Customer {
    uint64 id = 1;
    string short_name = 2;
    google.protobuf.Timestamp created_at = 3;
}

I would like to propose that Scan and Value methods be added onto the Wrapper and Timestamp Well Known Types. (And ideally a wrapped timestamp type as well, so that we can scan a nullable timestamp field in the db directly into a protobuf.)

I have created a proof of concept here to show that it can be done: https://github.com/veqryn/protobuf/tree/master/ptypes/timestamp The poc above uses a manually written Scan method that exists in a separate file from the generated file, but in the same directory/package. But gogo could use generated code instead if they wanted.

Looking at past issues, #108 is similar but the request in that issue is to Scan ANY protobuf message from bytes/json (and this is what the suggested repo does: https://github.com/travisjeffery/proto-go-sql ). My request is not to add a Scan and Value method onto any/all compiled protobuf messages (and also not to scan from bytes/json), but just to add it onto the Well Known Types that are most applicable and translatable to database fields: WKT Wrappers = sql.NullString, sql.NullBool, sql.NullInt64, etc (and hopefully sql.NullTime) WKT Timestamp = time.Time

jsnb commented 8 months ago

Is there an existing work around to achieve this kind of thing if you are using the timestamp type? I've got my protos generating my database schema it would be nice to have the go types can be used for scanning like you have described in the code snippet.