docs(architecture): proposal for fragments in Helix 5 architecture

[trieloff]

Please check this diagram:

https://github.com/adobe/helix-home/blob/b26ed5a01e0d8cd67a33c08d6f9aab72a612763f/docs/Franklin%20Architecture/Helix%205%20Stack%20with%20Fragments%20(AWS).png

1. There is a new S3 bucket for persistence, the Fragment Bus. Other than the content bus, it puts no constraints on the contained JSON format, so that JSON documents of any size, depth, schema, and complexity can be stored
2. The JSON from here is not served directly, but through the Fragments Service, a new CDN service at fragments.aem-fastly.page (with equivalents for Cloudflare). This service will provide arbitrary JSON filtering, joining, and projections. It will run at lower COGS and latency than the existing AEM GraphQL endpoint, but not be limited in terms of memory nor use new services.
3. The fragments served by the Fragment service can be consumed either by the client app directly (as this is a native mobile app, constraints around CORS, TLS, latency, etc. can be neglected) or by the customer's CDN, which can also apply an optional transformer service to provide arbitrary JSON format transformations.

[tripodsan]

LGTM, but I'm not sure about the name fragments for the bus (fragments should be anyways served as HTML IMO, so It's ok to use it a service name ;-).

but I'm ok with the bus name.

[stefan-guggisberg]

a new CDN service at fragments.aem-fastly.page

What about caching and invalidation?

[tripodsan]

@trieloff what about publishing?

[davidnuescheler]

i don't think that this is the right approach, and i am not sure what we are trying to achieve here...

we already have fragments with defined semantics in the product (aem-boilerplate) and this seems to be something that's orthogonal to those fragments.

[trieloff]

i don't think that this is the right approach,

fair enough.

and i am not sure what we are trying to achieve here...

We are trying to validate and strengthen the existing AEM headless architecture by giving them a place not just in the Edge Delivery Services marketecture (which is perceived by some as being detached from reality), but also in the Helix architecture diagram.

There is a desire to become a proper part of the Helix architecture without having to submit to any of the constraints that Helix typically imposes on content structure (such as "no nested blocks"), API format (such as "HTML is an API"), experience quality (such as "Lighthouse Score 100 is a baseline requirement"), experience architecture (such as "the use of client-side libraries such as React cannot be seen as a given, but is subject to their value and performance impact"), system architecture (such as "make sure that critical components can be provided by two redundant stacks", "validate the operational characteristics of each system component before putting it into production", "mind your innovation budget – there is only so much room for exciting tech"), or approach to product innovation (such as "validate ideas on projects first, before touching the core of the product").

To satisfy this desire without having to impact the operational properties of the rest of the system and to enable the headless team high degrees of autonomy, the proposal adds two components that touch the existing system only tangentially:

• the new Fragment Bus will be a separately managed S3 bucket, and does not have to share credentials with the existing buses
• the Fragments Service is a separate Fastly service that is not accessed by the Outer CDN, but customers will consume it through their BYO CDN or directly.

we already have fragments with defined semantics in the product (aem-boilerplate) and this seems to be something that's orthogonal to those fragments.

Indeed. We already have support for fragments, this proposal will add support for content fragments and experience fragments (this beginner-level explanation of the difference is a quick read and only 10% longer than the entire AEM developer tutorial).

I did not want to raise the expectation that the implementation of these more complex fragments would be a recommended practice for projects starting off the boilerplate or projects looking for effective experiences, high content velocity, or fast projects.