Implement grpc using protobuf or flatbuffers

[ghost]

Wishing to hear opinions..

I would prefer a faster API that can still work with json but I can use protobufs too.

I want to do some work on extending ponzu but hate the loose typing of json and http handlers.

If Flatbuffers with grpc were the basis then you can expose that also as a rest / graphql API for those that are cool with untyped.

It would also make building real apps on top of the headless CMS much easier. For example you can use a grpc Flatbuffers client that is code generated and then built complex GUI using gopherjs or flutter easily. Both support protobuf and Flatbuffers.

I am aware that this is a big refactoring. But I still think it's worth raising as ponzu might benefit.

[nilslice]

I love the idea of using protobufs over JSON, and still respond to http requests with JSON. I'm not very familiar with flatbuffers, but the general reason I have kept protobufs out of the project is that they add another compilation step, which I worry would be confusing or too complex to add on top of Ponzu (which is supposed to be a very simple system).

There are more tools (uber/prototool) which could be integrated and automate a lot of the setup but I'm not 100% convinced it's then right move.

What do you think?

[ghost]

True it does add another compile step. It can be configured as part of the cmd you already have to make a new user type in the system. Phase one is to gen the protobuf. Phase two is to run protoc Phase three is whatever pilosa needs more...

Also you can use the protobufs for a generic API. And the user generated types are within it as a binary array. That's what dgraph and cockroachdb do. It's also a reasonably option actually.

The management API should be a strongly typed protobuf though. It never changes and is not reliant on user types. I don't even know if you have a management API actually. I have not had much time.

The Uber tool looks way to green and I really think its YAGNI. Generating protobufs is easy. Maybe use the Uber tool later once the MVP works.

There is great support with gopherjs to work with protobufs and grpc now. It means the management web GUI can be gopherjs if you want. You can see it in the examples below. There is also a proper MDC wrapper now written for gopherjs. It's using vecty. https://github.com/agamigo/vecty-material

The reason I mention all this above is because you get golang strong typing all the way up from the protobufs to the controls on the GUI front end Sure it's not for everyone but it's pretty nice. And wasm IS going in golang 1.2 I hear so having your frontend written in golang is a good future proofed investment.

So in summary maybe start with only the management API first. Leave the user types until later ?

Some good examples:

https://github.com/johanbrandhorst/grpcweb-example

https://github.com/johanbrandhorst/grpcweb-boilerplate

Sorry about this long diatribe but I just wanted to get it all down.

All the libs above I have used. I am a strong typing proponent because I like compile time errors . As a project gets bigger finding the errors at runtime means spending so much time writting headless browser tests etc etc.

On Tue, May 29, 2018, 4:36 PM Steve Manuel notifications@github.com wrote:

I love the idea of using protobufs over JSON, and still respond to http requests with JSON. I'm not very familiar with flatbuffers, but the general reason I have kept protobufs out of the project is that they add another compilation step, which I worry would be confusing or too complex to add on top of Ponzu (which is supposed to be a very simple system).

There are more tools (uber/prototool) which could be integrated and automate a lot of the setup but I'm not 100% convinced it's then right move.

What do you think?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ponzu-cms/go-client/issues/10#issuecomment-392800165, or mute the thread https://github.com/notifications/unsubscribe-auth/ATuCwsIAqIxa_NvR-miQ-3sjiHwQjer2ks5t3V0GgaJpZM4URj2B .

[Globegitter]

Just my own opinion, I love the idea of a headless CMS with grpc API rather than JSON API (so for the frontend or any other backend that would consume data from that CMS). For the web there is also https://github.com/improbable-eng/grpc-web, but I am probably in the minority here and most people would prefer standard json based APIs.

But yeah grpc does add some additional complexity to a project, I do think once they have reached a certain size, or want wants the guarantees grpc it can really be worth it though.