docstore: Cannot search on nested protobuf generated structs

[eqinox76]

Describe the bug

A protobug generated struct can be Put and Get from the docstore but the Where search is not working.

To Reproduce

Minimal reproducible programm that writes one testType and tries to search for it. When changing the From property from date.Date to Date the search is working fine.

package main

import (
    "context"
    "fmt"
    "io"
    "time"

    "github.com/google/uuid"
    "github.com/samber/lo"
    "gocloud.dev/docstore"
    "google.golang.org/genproto/googleapis/type/date"

    _ "gocloud.dev/docstore/memdocstore"
    _ "gocloud.dev/docstore/mongodocstore"
)

type Date struct {
    Year, Month, Day int
}

type testType struct {
    Id   string
    From date.Date
}

func main() {
    ctx, c := context.WithTimeout(context.Background(), 10*time.Minute)
    defer c()
    coll := lo.Must(docstore.OpenCollection(ctx, "mem://Test/Id"))

    input := []*testType{{
        Id: uuid.NewString(),
        From: date.Date{
            Year:  2024,
            Month: 12,
            Day:   31,
        },
    }}
    for _, d := range input {
        lo.Must0(coll.Put(ctx, d))
    }

    read := func(iter *docstore.DocumentIterator) {
        for {
            read := &testType{}
            err := iter.Next(ctx, read)
            if err != nil {
                if err == io.EOF {
                    break
                }
                panic(err)
            }
            fmt.Printf("%+v\n", read)
        }
    }

    fmt.Println("Search Result:")
    read(coll.Query().Where("From.Month", "=", 12).Get(ctx))
    fmt.Println("All Documents:")
    read(coll.Query().Get(ctx))
}

results in:

Search Result:
All Documents:
&{Id:df3adf13-4903-489a-953d-130cb2fce8fb From:{state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:0xc000220160} sizeCache:0 unknownFields:[] Year:2024 Month:12 Day:31}}

Expected behavior

Query().Where("From.Month", "=", 12) must return one element.

Version

go version go1.23.1 linux/amd64

gocloud.dev v0.39.0 gocloud.dev/docstore/mongodocstore v0.39.0

Additional context

Doing the same with a local mongodb instance instead of the mem implementation acts the same way. In the mongodb i see the From property serialized as binary data which may explain this behavior:

Test> db.tests.find()
[
  {
    _id: '2e73454d-49ec-438d-a33f-9a053555d573',
    From: Binary.createFromBase64('COgPEAwYHw==', 0)
  }
]

[jba]

That's by design (possibly wrong design). We turn protos to binary blobs: https://github.com/google/go-cloud/blob/master/docstore/driver/codec.go#L135. Perhaps we should have used protojson.

We have no way to insert your own codec, but you could write your own driver and register it.

[eqinox76]

Ah, thanks for explaining. Serializing the messages via protojson is working fine. But having normal structs with partial protobuf messages is getting much more complex. I will have a look at implementing a kind of wrapper driver.

[jba]

We might add an option to serialize protos as JSON. (We can't do it by default, to preserve backwards compatibility.) You can file an issue for that.