search

zero-deps / proto

Lightweight and fast serialization library for Scala 2/3 based on Protocol Buffers with macros
MIT License
34 stars 3 forks source link

generate proto files #25

Closed tellnobody1 closed 2 years ago

tellnobody1 commented 2 years ago

to be used for other languages (dart, swift, js).

lxomb commented 2 years ago

This will be very convenient. However as per #27, it seems that the codec isn't fully compatible with protobuf specification, decoding non-packed version of repeated primitives is not supported.

tellnobody1 commented 2 years ago

In proto3, repeated fields of scalar numeric types use packed encoding by default.

I did not get why do you need full specification? Scala models will be source, proto files will be generated from Scala, not vice verse. If you want proto files to be main source of models then probably this solution doesn't fit the case. So main language here will be Scala and proto files are needed, for example, for frontend like JS/Dart/Swift/Kotlin. In that case it will be easier to have models in Scala code and not in proto files, because frontend depends on backend. If there are multiple backends then you can go with proto files and full specification, especially if some files is not under control.

lxomb commented 2 years ago

In proto3, repeated fields of scalar numeric types use packed encoding by default.

I did not get why do you need full specification? Scala models will be source, proto files will be generated from Scala, not vice verse. If you want proto files to be main source of models then probably this solution doesn't fit the case. So main language here will be Scala and proto files are needed, for example, for frontend like JS/Dart/Swift/Kotlin. In that case it will be easier to have models in Scala code and not in proto files, because frontend depends on backend. If there are multiple backends then you can go with proto files and full specification, especially if some files is not under control.

The other client uses protobuf2, and is writing unpacked encoding per default (and sadly I cannot change that). The specification requires implementations to support reading both encoding, so the behavior that non-packed encoding isn't read is surprising, at least. zero-deps/proto works perfectly fine with everything else in my project and I don't want to add another protobuf library solely for one field.

tellnobody1 commented 2 years ago

I think it is possible to add as a parameter for codec macros and annotation for fields. @shmishleniy @YuriyMazepin please take a look issue #27 if you have a time.

lxomb commented 2 years ago

I think it is possible to add as a parameter for codec macros and annotation for fields. @shmishleniy @YuriyMazepin please take a look issue #27 if you have a time.

Thanks for taking a look at this. By protobuf specification, decoding shall be always supported for both format, but encoding need not. So extra parameter or annotation is not needed if only decoding of non-packed version is to be supported I think.

tellnobody1 commented 2 years ago

Annotation still could be useful for decoding to not pass parameters to macros function. Like @NonPackedDecode.

lxomb commented 2 years ago

Annotation still could be useful for decoding to not pass parameters to macros function. Like @NonPackedDecode.

The point is, decoding both format should always be supported, at the same time, and even for interleaved onces, regardless options. Otherwise, sending protobuf to another client which simply decodes and encodes (with compatible protobuf file per specification) and sends back will cause the repeated field to be gone at Scala side, without any warnings or errors. This is dangerous and surprising.

tellnobody1 commented 2 years ago

no use case at the moment. could be done easily with scala 3 macros (generate string and write to file in runtime). or it is possible to generate code in target language directly without protobuf spec.