Generate protos for use in other languages

neoaggelos commented 4 years ago

Summary

Generate for use in other languages, e.g. C/C++/Python

Why do we need this?

So developers can marshal/unmarshal protocol buffers, create SDKs,

What is already there? What do you see now?

Protocol buffers generated for Go in pkg/ttnpb

What is missing? What do you want to see?

Protocol buffers for other languages

Environment

3.8.7

How do you propose to implement this?

From https://github.com/TheThingsNetwork/lorawan-stack/pull/2978#issuecomment-665134388:

Generate clean protos containing only external API - I would prefer to have (from repo root) internal/api and api, where we move all our current protos to internal/api and then generate clean, vanilla protos (.proto files) in /api (perhaps including any possible proprietary APIs as well)
Contain generated protos in SDKs (generated protos are then the starting point of an SDK - so for a language X, an MVP of a X-SDK is just a collection of generated "clean" external protos in language X)

Can you do this yourself and submit a Pull Request?

No

rvolosatovs commented 4 years ago

Well, the most important point here is that, apparently, compiling our current protocol buffers is not a straight-forward process for users. It is not realistic to attempt to provide compiled protocol buffers for all languages supported (officially or otherwise) by protoc, so we need to make the compilation process as simple as possible - hence, the suggestion is to generate clean, minimal protocol buffers (i.e. .proto) files. In my opinion the wording in the OP is not completely clear on this point. Please see https://github.com/TheThingsNetwork/lorawan-stack/pull/2978#issuecomment-665134388 for context and the proposal in it's entirety.

johanstokking commented 4 years ago

Are you proposing a) to have all protos that are used by lorawan-stack in internal/api, and generate clean protos (of external API) to api or b) that all external API is in api and only internal API is in internal/api, i.e. that lorawan-stack uses protos generated from api too? I think option (b) is pretty problematic.

In any case, having clean, vanilla protos with no dependencies other than standard Protocol Buffers stuff in api would be ideal. From there, we can generate to some popular languages, but if developers need their own for some exotic languages, they would be able to do that easily.

rvolosatovs commented 4 years ago

a.) In v4 we could try b.)

This is also why I suggest to have a tool, which would let us selectively choose entities to "export", we could also just separate these by filename, of course, or maybe write yet another protoc plugin, which would let us export not only specific structures/services, but also specific fields/methods, while keeping other ones hidden.

On Fri, Jul 31, 2020, 13:11 Johan Stokking notifications@github.com wrote:

Are you proposing a) to have all protos that are used by lorawan-stack in internal/api, and generate clean protos (of external API) to api or b) that all external API is in api and only internal API is in internal/api, i.e. that lorawan-stack uses protos generated from api too? I think option (b) is pretty problematic.

In any case, having clean, vanilla protos with no dependencies other than standard Protocol Buffers stuff in api would be ideal. From there, we can generate to some popular languages, but if developers need their own for some exotic languages, they would be able to do that easily.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/TheThingsNetwork/lorawan-stack/issues/3006#issuecomment-667046548, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADCIAUMULQYAKJ73GDELFOLR6KKGBANCNFSM4PN227JQ .

htdvisser commented 2 years ago

Removing assignments and bumping back to triage for re-assignment and prioritization.

Maybe this should be merged into a new "SDK" umbrella issue?

cc: @NicolasMrad

TheThingsNetwork / lorawan-stack