golang / protobuf

Go support for Google's protocol buffers
BSD 3-Clause "New" or "Revised" License
9.74k stars 1.58k forks source link

protojson fails to unmarshal *timestamppb.Timestamp #1532

Closed Codewolf closed 1 year ago

Codewolf commented 1 year ago

What version of protobuf and what language are you using? libprotoc 3.21.9 protoc-gen-go-grpc v1.3.0 go version go1.19.3 darwin/arm64

What did you do? Create a test Message containing only relevant data

syntax = "proto3";
import "google/protobuf/timestamp.proto";
option go_package = "sandbox/protobufstuff";
message TestMessage {
  string uuid = 1;
  string rootTeamId = 2;
  google.protobuf.Timestamp contractStartDate = 3;
}

Create a testing script to unmarshal the a json string into the protobuf.

package main

import (
    "fmt"
    "google.golang.org/protobuf/encoding/protojson"
    "sandbox/protobufstuff/sandbox/protobufstuff"
)

func main() {
    fmt.Println("Start Unmarshalling")
    j1 := []byte(`{"rootTeamId":"062bffb0-adcf-4366-9ee8-e94d473f5d4d","uuid":"b9b383f8-4606-42eb-bba4-f213f7c0f197"}`)
    message1 := &protobufstuff.TestMessage{}

    err1 := (protojson.UnmarshalOptions{
        DiscardUnknown: true,
    }).Unmarshal(j1, message1)
    if err1 != nil {
        panic(fmt.Errorf("protojson unmarshal : %w", err1))
    }
    fmt.Println("Message 1 Unmarshalled Successfully")

    j2 := []byte(`{"contractStartDate":{"nanos":0,"seconds":1},"rootTeamId":"062bffb0-adcf-4366-9ee8-e94d473f5d4d","uuid":"b9b383f8-4606-42eb-bba4-f213f7c0f197"}`)
    message2 := &protobufstuff.TestMessage{}

    err2 := (protojson.UnmarshalOptions{
        DiscardUnknown: true,
    }).Unmarshal(j2, message2)
    if err2 != nil {
        panic(fmt.Errorf("protojson unmarshal : %w", err2))
    }
    fmt.Println("Message 2 Unmarshalled Successfully")
}

What did you expect to see?

Start Unmarshalling
Message 1 Unmarshalled Successfully
Message 2 Unmarshalled Successfully

What did you see instead?

Start Unmarshalling
Message 1 Unmarshalled Successfully
panic: protojson unmarshal : proto: syntax error (line 1:22): unexpected token {

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment? I have referenced https://github.com/golang/protobuf/issues/1433 as well as reading https://developers.google.com/protocol-buffers/docs/gotutorial, otherwise the code above was written in a clean project made specifically for this error

puellanivis commented 1 year ago

Timestamp is not encoded as a message with seconds and nanos in JSON, instead it is encoded with RFC 3339:

Timestamp : string : "1972-01-01T10:00:20.021Z" : Uses RFC 3339, where generated output will always be Z-normalized and uses 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted.

https://protobuf.dev/programming-guides/proto3/#json

Codewolf commented 1 year ago

Timestamp is not encoded as a message with seconds and nanos in JSON, instead it is encoded with RFC 3339:

Timestamp : string : "1972-01-01T10:00:20.021Z" : Uses RFC 3339, where generated output will always be Z-normalized and uses 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted.

https://protobuf.dev/programming-guides/proto3/#json

huh, my bad, i was so focused on where it was erroring i completely failed to notice that. The marshaller i use must be putting out the incorrect format. Thanks @puellanivis