[Question] About Gun

[yordis]

That project looks good but it depends on :gun which doesn't play nice with other dependencies.

Hey peeps, I am curious to learn from your experience about such a statement. Do you mind expanding fo what was the issue with :gun?

Thank you in advance.

[the-mikedavis]

Sure thing!

Gun 1.3.x (currently the latest stable release of gun) has very specific dependencies on cowlib and so does cowboy, so any time you want to use cowboy and gun in the same project, you'll need to take on an override. That was more of a problem with Slipstream though which was more likely to be used in a Phoenix project (Phoenix uses cowboy as the underlying HTTP/WebSocket server). You can't publish to hex with an override, so any hypothetical library built on Spear that may include cowboy couldn't be published. Gun's 2.0.0 branch has had pre-releases for a while but it hasn't had the full 2.0.0 release yet, so it's likely you'd run into conflicts if you use Spear along with other dependencies that depend on gun's 1.x line. See also: https://github.com/nerves-hub/nerves_hub_link/pull/68#issuecomment-794616197

I don't mean to slam gun: I think it's a really great library. It's just hard to use in Phoenix projects at the moment which makes it a less-than-ideal choice for use in an Elixir library.

[yordis]

Ah!!! I see I see the problems ahead, I keep forgetting about such problems until I ran into them.

Thank you so much for the explanation.

[yordis]

@the-mikedavis the new elixir-grpc/grpc allows to use Mint over Gun.

For the sake of alignment in the ecosystem and helping each other.

Is it prudent to give it a second try to the gprc package? Are you open to it if the solution is good? (whatever that means to you)

[the-mikedavis]

Oh cool, I'm glad to see grpc pick up Mint support, I think that will be a big improvement for using gRPC in Elixir 🚀

I took a look at the PR that added Mint support. The two things I was mostly looking at were the handling of concurrent requests, so that Spear can append many events and stream many subscriptions all though one connection process, and sending very large messages/streams. The concurrent stuff looks good. The streaming support is better in Spear though: from what I can tell, grpc will send a message to the connection process for every message it wants to send in a stream. In Spear we unwrap streams lazily within the connection process and fit as many messages into a frame as possible. This ends up taking only one Erlang message since we just hand over the stream to the connection process vs one Erlang message per gRPC message. I would worry about the performance of many appends suffering but this would need to be tested. Also see #3 and https://github.com/sneako/finch/pull/158.

Looking around the rest of the codebase though, there are a few reasons I don't want to add in grpc:

• It bundles client and server functionality into one library. Spear only needs a client and the rest would just be bloat.
• grpc has non-optional dependencies on a bunch of rather large libraries like cowboy and their fork of gun's 2.0 release line: https://github.com/elixir-grpc/grpc/blob/7bd40696582fb5e6640ea846d4f6989e6d754fa4/mix.exs#L42-L47. So even though you can use Mint at runtime now, they still depend on gun which is a problem.
• The Mint adapter uses an extra process for response streams which isn't necessary for Spear: https://github.com/elixir-grpc/grpc/blob/7bd40696582fb5e6640ea846d4f6989e6d754fa4/lib/grpc/client/adapters/mint/stream_response_process.ex

Mainly I want to keep the dependency tree for Spear very small. The existing spear dependencies should be compatible with grpc's dependency list. It would be a really big change in Spear to switch to grpc so I would like to only do that if there's a big problem with using Mint directly

[yordis]

Is it worth connecting with them and discussing your pain points?

I am trying to allocate some budget and energy to create a better foundation across the ecosystem. Some points could be addressed and improved in the gRPC package.

[the-mikedavis]

I think that Spear is too niche of a use-case to justify spending effort making the grpc integration perfect for it. Spear wants really low-level control over gRPC details so even if grpc satisfied all of my nitpicks I would probably still not use it in an effort to keep really fine-grained control over the connection and also keep the dependency tree small.

I haven't used gRPC outside of this client or in other languages so I'm not sure what's missing for a really nice experience. I think it is probably best to survey what typical gRPC users want to see in the ecosystem.