[AP proposal] Toward a more unified UX: BFF 💕

[andreaTP]

Problem statement

The integration of the Enterprise option of RHOSAK in UI, SDKs and CLI has shown that there is a margin for improvement in how we expose and consume our services APIs.

More specifically:

• the current services are offering the "bare functionality" (as they should) without providing many helpers
• the data aggregation logic is scattered, duplicated, and possibly even diverging in various codebases (especially UI and CLI)
• the integration of the various services is done in multiple places
• to achieve a desired functionality through the REST APIs the end user is exposed to the entire complexity of the underlying infrastructure

One of the patterns applied to improve when those issues arise is BFF 💕 .

Backend For Frontend

PROS:

• aggregate business logic to provide functionalities to users
• exposes endpoints for direct consumption from UX
• can be stateless (less maintenance, easy scaling up/down)
• UI and CLI need to know much less about the underlying infrastructure and can be thinner
• testing of the user-facing components can be done by mocking a single service instead of multiple APIs
• centralized patterns can be applied to a single component, such as rate-limiting etc.

CONS:

• an additional service to operate in the backend
• maintenance of a shared codebase for multiple teams (UI and DevExp)

Current Architecture

With BFF

Open questions:

• how do the current REST APIs fit into the desired User Experience?
• should we keep exposing the "bare functionality" or should it become a "technical detail" for the end user?

the idea has been conceived by speaking with @riccardo-forina and shared with the DevExp team( @dimakis ); we are convinced, at this point, that the advantages are outweighing the disadvantages but we are happy to receive early feedback before opening a formal AP.

[andreaTP]

cc. @EricWittmann @MikeEdgar @machi1990 @tombentley @lburgazzoli @emmanuelbernard @miguelsorianod @shawkins for feedback, and feel free to extend the visibility to anyone that might be interested in chipping into the conversation!

[EricWittmann]

Interesting approach, for sure. I'm sympathetic to it. Can you give some concrete strawman examples of the functionality this BFF would provide?

[machi1990]

cc. @EricWittmann @MikeEdgar @machi1990 @tombentley @lburgazzoli @emmanuelbernard @miguelsorianod @shawkins for feedback, and feel free to extend the visibility to anyone that might be interested in chipping into the conversation!

Adding @pb82 as well.

[tombentley]

Stupid question: What are the backends you envisage? Generally it would really help (me at least) if you could give concrete examples for each of the current problems you identify.

[andreaTP]

I'll coordinate with @riccardo-forina and @dimakis to provide a concrete example based on the recent experience of Enterprise 👍

[lburgazzoli]

I'm very sympathetic to the idea as the topics pops up every now and then in the RHOC team as it is very complicated i.e. for the UI to deal with all the low building block that often cross the single service boundaries.

[EricWittmann]

I'm very sympathetic to the idea as the topics pops up every now and then in the RHOC team as it is very complicated i.e. for the UI to deal with all the low building block that often cross the single service boundaries.

+1 - I think this problem is likely to get worse as we continue to try and integrate services more tightly in the UI.

Do you imagine these BFF APIs to be available to customers as well?

[lburgazzoli]

I'm very sympathetic to the idea as the topics pops up every now and then in the RHOC team as it is very complicated i.e. for the UI to deal with all the low building block that often cross the single service boundaries.

+1 - I think this problem is likely to get worse as we continue to try and integrate services more tightly in the UI.

Do you imagine these BFF APIs to be available to customers as well?

If done right, they could be a differentiator so I'd say at some point the could be made available to everyone.

[riccardo-forina]

I actually have a different view for this. I see the existing APIs as the low level, non-MAS specific ones available to all upstream users to use as it works best for them. The BFF on the other hand should be heavily MAS specific as it needs to deal with concerns that us and us only (e.g. all the quota and billing bits)

[dimakis]

Stupid question: What are the backends you envisage? Generally it would really help (me at least) if you could give concrete examples for each of the current problems you identify.

Taking an example from Hybrid (Enterprise) the displaying of Hybrid cluster names and that cluster's Id during the CLI's list or during create kafka commands. This is currently two separate calls, one to KFM for cluster details and the other to OCM just to fetch the cluster name. To grab cluster capacity a further call to KFM is needed per individual cluster.

Furthermore, with the current approach, this logic would be duplicated in the CLI and UI. Even though at present, it is not really possible to do this well in the UI, as OCM doesn't provide a JavaScript/TypeScript SDK. A fetch to URL or rolling out our own generated SDK of OCM are the kind of options that have been discussed as a result.

There are further bits of duplicated logic when querying AMS for the user's quota etc. too.

[andreaTP]

What are the backends you envisage?

@tombentley what @dimakis mentioned above ☝️ and also from @riccardo-forina experience. Creating a Kafka Instance when Enterprise is active roughly resemble to:

• getting information for the current account (multiple calls to get quotas, entitlements, current usage etc. etc.)
• gettting cloud providers/regions and more available options from fleet manager
• obtain information regarding the currently deployed instances of Kafka
• combine all the information above to know the available options for the user

Looking at the code, you can notice that all the merging of the data to understand quotas/availability/possible options is done both in CLI and UI. Also, just to show the "Create Kafka" screen (or show the available options in CLI interactive mode), we are approx in between 3 to 10+ API calls and this is starting to impact performance too.

Having a server side component that maps one UI/CLI concern to one API call would be extremely beneficial and can be subsequently further optimized with caching and more.

Do you imagine these BFF APIs to be available to customers as well?

@EricWittmann this is a super interesting discussion, but I would feel more comfortable in having it after we decide to have this component in place for internal usage first. Realistically speaking, covering the needs of CLI/UI and customers with a single API seems like a stretch and highly subject to premature over engineering, but I might be wrong ... I propose to re-evaluate this point with the BU after we have a first implementation in place.

[tombentley]

Is there one BFF for each service?

[tombentley]

(BTW thanks for the clarifications so far!)

[andreaTP]

Is there one BFF for each service?

NO, I think this point has been made clear by the visuals added at the top, there should be one BFF for the entire platform. It's extremely important that this is clear, please 🙏 , let me know if it's not the case @tombentley

[tombentley]

/create ap

[tombentley]

/create ap

[bf2-arch-bot[bot]]

Closing following creation of AP-19 @andreaTP, please write your content in _ap/19/index.adoc and open a PR for AP acceptance.

[emmanuelbernard]

Thanks for this proposition. I have the following questions I would like to see addressed:

• the BBF will call into the data plane admin API: how is this going to work with Enterprise Plan with VPC isolation (I'm assuming the customer would get access to this VPC from where it is calling the CLI).
• because it becomes a public API used by the UI and the CLI, how can we make it not a supported API. I'm concerned about the proliferation of APIs though I'm sympathetic to the BBF pattern.
• What is the expectation, one BBF for all key exposed services (Service Registry, Connectors, Kafka etc) or one per such service? The drawing only focus on Kafka usage.
• what makes it not possible for our API to address the UI and CLI needs (esp why isn't kas-fleetmanager API not shielding the quota management aspect). The reasoning is that if we need to do it why wouldn't a user of the API need it too?
• is the CLI and UI the same calls? The classic BBF patterns tends to promote one BBF per front end (one of CLI, one for UI, etc).

[andreaTP]

Hi @emmanuelbernard thanks for chipping into this issue, much appreciated! I might need some help to fully answer all the questions and nothing is yet set in stone but let me try to go one by one and keep the ball rolling:

the BBF will call into the data plane admin API: how is this going to work with Enterprise Plan with VPC isolation (I'm assuming the customer would get access to this VPC from where it is calling the CLI).

I would need to better check all of the invariants, but the options should be:

• deploy the BFF in the data-plane
• deploy a proxy to the BFF in the data-plane
• spin a BFF instance in the background for the CLI
• how are the discussions for moving data-plane components into the control-plane going?
• how would we handle a similar situation(when this applies) with regard to e.g. monitoring/observability?

because it becomes a public API used by the UI and the CLI, how can we make it not a supported API. I'm concerned about the proliferation of APIs though I'm sympathetic to the BBF pattern.

I can't think of big concerns in marking the BFF API as "supported" as long as we are able to quickly iterate for internal needs. On this subject, we should consider the currently exposed APIs and I think it's fair to say that someone outside of RH would have a very thought time trying to directly use them. If we do a good job in API design the BFF API might become the only one needed by users and we can eventually consider marking the others as "internal". But, staying pragmatic, with this proposal, we are going to increase only by 1 the number of APIs, but with dramatic benefits from the current consumers(UI and CLI).

What is the expectation, one BBF for all key exposed services (Service Registry, Connectors, Kafka etc) or one per such service? The drawing only focus on Kafka usage.

We are currently thinking about 1 BFF to rule them all 💍. But open to further discuss on this topic.

what makes it not possible for our API to address the UI and CLI needs (esp why isn't kas-fleetmanager API not shielding the quota management aspect). The reasoning is that if we need to do it why wouldn't a user of the API need it too?

The current APIs are addressing all of the UI and CLI needs, at the cost of multiple calls and crossing data. This is anyhow an interesting question, the short answer is that the information needed by the clients is crossing the boundaries of a single service. E.g. we need information from account_management that is not available, at the moment, in fleetmanager at all. Breaking the current separation of concerns between services IS indeed a way to achieve the same result at the cost of falling into a number of bad practices. The proposal of a BFF serves, as well, to identify one specific component that will take care of crossing information from various systems while keeping them autonomous and specialized.

is the CLI and UI the same calls? The classic BBF patterns tends to promote one BBF per front end (one of CLI, one for UI, etc).

I have the(personal) feeling that starting with a single BFF we will be already improving the situation in the short to mid term. But a full and unbiased answer to this question cannot be done without including BU in the conversation and getting a grasp of the desired UX for UI and CLI in the longer term. (happy to gather additional feedback and be directed to relevant stakeholders!).

[emmanuelbernard]

By suppored API I mean we never break it which seems a bit counter to the idea of the BBF flexibility you are looking for.

[emmanuelbernard]

@andreaTP let's discuss it at the arch call tomorrow