msoucy / dproto

D Protocol Buffer mixins to create structures at compile time
Boost Software License 1.0
37 stars 16 forks source link

support serialization to/from human-readable, text-based format #71

Open timotheecour opened 8 years ago

timotheecour commented 8 years ago

http://stackoverflow.com/questions/18873924/what-does-the-protobuf-text-format-look-like ref: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format#TextFormat.Parser.WriteLocationsTo.details

I only see json and binary serialization (wire format)

the text format is more convenient than json and is commonly used would be nice to have it for compatibility

msoucy commented 8 years ago

I've never seen this form of serialization before, it's not mentioned in the docs that I've read. Do analogues exist for other first-party implementations?

timotheecour commented 8 years ago

well it's mentioned and described in the above links [including official docs] it's very commonly used, eg when writing tests, or when writing queries in text proto format again, it's a more readable syntax than json

at least proto->string direction should be easy.

there are 2 apis for this direction: toStringShort (output in 1 line) toStringLong (output in multiple lines with indentation)

and only one for other direction

timotheecour commented 8 years ago

eg: this type of file is common: https://github.com/BVLC/caffe/blob/master/models/bvlc_reference_caffenet/solver.prototxt

timotheecour commented 7 years ago

@msoucy ping on this; if it's not on your roadmap, would you have any pointers to get started?

msoucy commented 7 years ago

It's not on my roadmap... when I started dproto the format didn't even exist.

The main printing code would probably be added to the mixin in attributes.d similar to the serialization code... Though the main logic might be complex enough to warrant being put into another file. The serializeProto function seems like it would have a decent example of how to start adding the code.

timotheecour commented 6 years ago

standard text format is actually good but could be better, I propose the following, which handles repeated fields differently in a more json-like way by only putting the name of a repeated field once (or 0) instead of once per repeated element

# dub.prototxt
name: "test1"
sourcePaths: [ "source1", "source2"]
dependencies: [
  { name: "foo", path: "some/path/foo"}
  { name: "bar"}
]

instead of:

name: "test1"
sourcePaths: "source1"
sourcePaths: "source2"
dependencies: { name: "foo", path: "some/path/foo"}
dependencies: { name: "bar"}

Essentially, this is more DRY.

timotheecour commented 6 years ago

IIRC i have used a workaround based on using protoc which is not great because it requires shelling out