gRPC example

[sudhirj]

Is seems to be possible to run gRPC services on Fly, but I'm not entirely sure about the details.

I'd assume the following needs to be done

• use the TLS handler for SSL termination on Fly
• initiate the gRPC server without authentication, the TLS termination on Fly ought to be enough
• initiate clients with SSL authentication

Will try this out and confirm if everything works.

[mrkurt]

This should work! We don't have a good gRPC demo yet, so we're super curious what you come up with.

The only blocker I can think of might be http2, the TLS negotiation doesn't include the ALPN extension that indicates HTTP2. Depending on the gRPC library, that might be a problem.

Also if you're interested in us paying you to build a gRPC demo, we're working on this "get paid for examples" program right now! Feel free to email me (kurt at fly.io) and we can work it out.

[sudhirj]

Will try it out... based on examples I’ve seen with AWS Application Load Balancers, the gRPC servers seem to be pretty optimistic and assume H2. Is there a technical problem with adding the ALPN header if necessary?

[mrkurt]

There's no technical problem, we just haven't exposed it in the config yet. We can get that figurd out.

[sudhirj]

@mrkurt I've got a gRPC running on Fly, no issues so far. The configuration of

[[services]]
  internal_port = 54321
  protocol = "tcp"

  [[services.ports]]
    handlers = ["tls"]
    port = "443"

works fine. The Fly load balancer / router is handling the SSL termination and non-TLS forward to the app just correctly. And because it's running on the TLS->TCP handler there's no HTTP logic interfering with the wire formats.

Will start the write-up. I want to double check streaming as well, that might be on people's minds.

Do we need to do separate checks for custom domains? I'd assume that all the same logic for TLS app.fly.dev will apply for custom domains as well.

Will also do a check for TCP pass-through, though I don't think it's all that useful.

[mrkurt]

Oh that's great.

We can actually enable gRPC health checks pretty easily, so if you're cool with it let's plan on that?

[sudhirj]

TCP health checks work fine, from what I can see. If you're talking about https://github.com/grpc/grpc/blob/master/doc/health-checking.md it might be a bit premature, unless Fly is going to add special dashboards and monitoring output for gRPC. There's also some extra config to tell Fly which service(s) to check health for, which might be fiddly.

[sudhirj]

I've put a first draft on https://github.com/sudhirj/grpc-fly-example

[mrkurt]

We're playing with this! ghz is pretty f'n cool.

[sudhirj]

@mrkurt Cool, I need to add a gRPC-Web Proxy example as well, the service as it is can't be accessed from any web browser. Fly doesn't currently support the sidecar or Docker Compose concept in any way, right? If there's a companion container (proxy / logging collector / DB connection pool) it needs to be a separate app for all intents and purposes?

[mrkurt]

That's right! For some stuff, it makes sense to just run one container/image.

[sudhirj]

@mrkurt I've updated https://github.com/sudhirj/grpc-fly-example with web-proxy examples and instructions as well, so it's now a complete guide to running a fully working gRPC system that works on browsers as well.

I'm also running my own gRPC service and browser proxy for an app I'm building, and can confirm that works great so far. The examples and fly configs are based off my own apps.

Let me know if there's anything else you think should go into the example, or if anything should be taken out?

[codepope]

@sudhirj Some notes:

"Create a new Fly app using flyctl apps create - don't overwrite the configuration at this point, the special configuration that gRPC requires has been set in fly.toml and explained below."

First, Latest version of flyctl supports "flyctl init ..." (flyctl apps create is deprecated)

You don't explicitly say how to resolve this with the downloaded app. If you say no to over-writing the file, flyctl exits.

I have a little script here -

out=$(mktemp) while IFS= read -r line do echo "$line" >> $out if [[ $line =~ ^app\ =\ . ]]; then break fi done < fly.toml while IFS= read -r line do if [[ ! $line =~ ^#. ]]; then echo "$line" >> $out fi done < template.toml cp $out fly.toml

- Rename your example fly.toml to template.toml, let the user run flyctl init - then run the script as import.sh and it'll do the job of sticking the two files together.

Getting the app name - flyctl info -n - returns just the app name

[codepope]

@sudhirj Actually.... after consulting inhouse....

csplit -s fly.toml '/app = .*/1'
cat xx00 > fly.toml
sed '/^#.*/d' template.toml >> fly.toml
rm xx0[01]

The supershort version of that script.

[codepope]

More notes. You don't need the app name if you are deploying from the directory where the fly.toml is. You can just say flyctl deploy and flyctl will find and use the fly.toml file.

[codepope]

Ah. The gRPC-Web section. (1) it doesn't let the reader know the dockerfile will need to be changed to point at the backend you've creates. (2) If you just deploy it you get no apparent examples of doing queries through the grpc-web mechanism. The grpc-web docs are somewhat overwhelming and I'm not sure the examples are in sync. Can you expand a bit upon that? If you browse to it, it just says gRPC requires HTTP/2

[sudhirj]

Will change the fly.toml to a template, that makes sense.

Will update the gRPC-Web section to talk about changing the Dockerfile.

Wondering how to explain the gRPC-Web section with an example... actually showing it in action requires a full webpack / package.json based app that needs to be installed and then run in a browser, which seems pretty heavy for this example. Will see if I can curl call that demonstrates it, that should be possible.

I need to transfer the repo over to Fly as well, shall I make the changes and do that or do you want me to do that right away and raise a PR?

[mrkurt]

Go ahead and make changes and then we can worry about transferring. Github makes transferring repos a little harder than it needs to be.

Is that client app already developed? If it is, you can just drop the whole thing in /ui/ or something and I'll make it build + serve as part of the app.

[sudhirj]

I have other client apps, but it's easy enough to make one. The issue is it needs the whole node_modules + download the internet to work. Will check if we do a normal request first.

[mrkurt]

Yeah that's fine, if it's easy to whip up. If it's hours of work don't bother. We have a pretty good set of static builders that run as part of the Docker process for that sort of thing.

[sudhirj]

@codepope Have added the import scripts and updated the commands to just flyctl init and flyctl deploy. Will figure out a check for web proxy.

[sudhirj]

@codepope @mrkurt I've added an example browser client in as well, just a basic page that outputs to console log. This will allow a full check of the web proxy as well.

I've also deployed the apps named in the examples myself, so even if folks run any of the commands without deploying anything they'll hit the apps I've deployed and actually see outputs.

Also did one round of editing and fixed a few typos.

[mrkurt]

Nice! Go ahead and transfer the project to my GitHub user (mrkurt) when you're ready. I can add you as a collaborator after it's moved to our org so you can keep tweaking.

[sudhirj]

@mrkurt I've started the transfer, you should see a request.

[mrkurt]

@sudhirj ok the readme is showing here, have a look and see if you want to make any changes before we push this out next week: https://fly.io/docs/app-guides/grpc-and-grpc-web-services/

[sudhirj]

Have added links to make it easier to jump straight to the code and raised a PR: https://github.com/fly-examples/grpc-service/pull/1

The styling on fly.io/docs doesn't currently differentiate between <code> elements that are also links, so they both look the same—this is an issue because a lot of the links are filenames or tool names that are in backticks. Should I change them to normal text or can you add a style cue?

[codepope]

Merged...

[sudhirj]

Can someone take over the grpc-web-proxy-test and grpc-test test apps from me? I've left them running so people can run the examples without having to install anything themselves.

[mrkurt]

Yes, good call. Give me a few.