feat: @farcaster/js should run in browsers

[varunsrin]

What is the feature you would like to implement?

Get @farcaster/js compiling in browsers

Why is this feature important?

Single-page and stand alone client applications will be able to interface directly with Hubs without a server.

Will the protocol spec need to be updated??

No

Additional context

• Can @farcaster/js just use grpc-web instead?
• Can @farcaster/hubble run a server that runs both grpc-js and grpc-web?
• Can @farcaster/js compile in browsers end to end?

[pfletcherhill]

Related other tickets to consolidate:

• https://github.com/farcasterxyz/hubble/issues/424
• https://github.com/farcasterxyz/hubble/issues/497
• https://github.com/farcasterxyz/hubble/issues/473
• https://github.com/farcasterxyz/hubble/issues/494

[kcchu]

My two cents:

1. I am uncertain about the use case of calling Hub RPC from web browsers. The RPC methods are primitive compare with Merkle API (e.g. Hub RPC don't handle threading). A web browser client couldn't do much thing without talking to a higher-level API server (e.g. Merkle API, Farcaster Indexer).
2. Client-Hub communication makes sense if the plan is to implement most functionalities needed by a client in Hub RPC.
3. And alternative approach to (2) is releasing a separated Farcaster API services. Hub remains as a service that is primarily for distributing messages and syncing with the peer-to-peer network. The Farcaster API talks to a Hub and expose higher level API for clients (it may implement add-on functions on top of Hub like searching).
4. Even if grpc-web is to be implemented in Hub, RPC between Hubs should stay on standard gRPC because it supports the most programming languages and much more efficient.
5. There is no Typescript implementation of grpc-web proxy. A solution is embedding a Envoy proxy in the docker image to serve the grpc-web port.

[pfletcherhill]

Supporting web browsers isn't an immediate problem, but in the medium term it lowers the barrier to entry for community members looking to make something using farcaster, which is good. I imagine some devs will want very basic access and choose to use the hub RPC for reading/writing and others will use more feature-rich APIs or build their own backends for indexing data. There's certainly an opportunity for community members to make their own indexing services and APIs on top of hubs for many use cases, similar to Alchemy's various APIs for ethereum.

We should support both grpc and grpc-web simultaneously using a proxy (I made it work with Envoy, as you mentioned, though there may be other options too). So hub-to-hub communication will continue to use gRPC directly.

[varunsrin]

we should see whether farcaster/js works by default in serverless platforms like vercel. if it does, we can punt this to a later milestone

[pfletcherhill]

High-level goal:

Create a new package called @farcaster/hub-web that exports the same methods as @farcaster/hub-nodejs except with a grpc-web client instead of a grpc one that works in browsers.

Work to be done:

• Create new @farcaster/hub-web package in the monorepo
• Generate grpc-web compatible client from the existing rpc.proto schema and export it from the new package. If using ts-proto, we can use the outputClientImpl=grpc-web option (see documentation).
• Wrap the grpc-web client implementation in promises and results (see client.ts in utils for reference)
• We want to re-export all of @farcaster/utils from the new package, but utils right now depends on @farcaster/protobufs which depends on @grpc/grpc-js which is not supported in the browser. So we need to refactor utils to not depend on grpc-js. One option would be to remove any services generated code from the protobufs package (see the outputServices=false ts-proto option) and add generated grpc-js code to the @farcaster/hub-nodejs package. That way the nodejs package has the grpc-js client and the web package has the grpc-web client.
• Refactor the hubble app to either use the @farcaster/hub-nodejs package or generate its own server and client grpc-js implementations from the rpc.proto schema, since right now those are imported from the protobufs package (see server.ts and hubble.ts).
• Bonus points for merging @farcaster/utils and @farcaster/protobufs into a single @farcaster/core repo, since they'll always be used together and should support all environments after the refactor.
• Further bonus points for developing a nice pattern for sharing protobufs schemas between packages in a monorepo.
• Add a grpc-web proxy to the hub deployment. See grpc-web docs for envoy example.

[dawsbot]

It appears this is solved, yet the readme states that it is not

I've supplied a PR to remove that! Let me know if this looks right 🙏 https://github.com/farcasterxyz/hub-monorepo/pull/1865