philpearl / plenc

Go encoding based on protobuf, but hopefully easier to deal with
MIT License
28 stars 2 forks source link

Standard slice encoding #9

Open 3052 opened 11 months ago

3052 commented 11 months ago

Using this code:

package main

import (
   "fmt"
   "github.com/philpearl/plenc"
)

type UserField struct {
   Key string `plenc:"1"`
}

type header struct {
   UserFields []UserField `plenc:"6"`
}

func main() {
   head := header{
      UserFields: []UserField{
         {Key: "key"}, {Key: "key"},
      },
   }
   out, err := plenc.Marshal(nil, head)
   if err != nil {
      panic(err)
   }
   fmt.Printf("%q\n", out)
}

I get this result:

"3\x02\x05\n\x03key\x05\n\x03key"

If I run similar code with github.com/golang/protobuf/proto, I get:

"2\x05\n\x03key2\x05\n\x03key"

I think the difference is because of this:

The big difference is that plenc uses its own encoding for slices of types that are implemented with WTLength. Plenc introduces a new wire-type for these - WTSlice (3). The Tag byte is followed by a unsigned varint containing the number of elements in the slice, then each element is encoded with its length as an unsigned varint then the element encoding.

https://github.com/philpearl/plenc#is-this-protobuf

I think its good that Plenc offers improved encoding, but it seems that both client and server would need to be using Plenc. If client was using Plenc and server not, then it seems server would reject the Plenc encoded message. If this is the case, then I think the standard encoding should also be offered, either via another function, or an option.

3052 commented 3 days ago

as a workaround I wrote this module that uses the standard encoding:

package main

import (
   "154.pages.dev/protobuf"
   "fmt"
)

func main() {
   data := protobuf.Message{
      6: {
         protobuf.Message{
            1: {protobuf.Bytes("key")},
         },
         protobuf.Message{
            1: {protobuf.Bytes("key")},
         },
      },
   }.Marshal()
   fmt.Printf("%q\n", data)
}