Global sourcemaps for microfrontend support

mortargrind commented 1 year ago

Feature request description

Hi,

We have a frontend application where we have “federated” microfrontends: essentially each microfrontend is managed by a different squad and can release new versions independent to each other. These microfrontends are not seperated via page/route boundaries so at any point in time there might be multiple microfrontend apps present on the page (I will call microfrontends “MFE”s throughout this post)

The app itself is using DataDog’s RUM SDK and “the shell” frontend is initializing it. We want to see each error thrown by each part of the application to have human readable stack traces (since we are minifying the code, they are not human-readable by default).

Each MFE is completely self-managed (including their teams, repos and build pipelines)
DataDog RUM SDK can only be initialized once and there can be only a single app version value (which is the shell frontend’s version)
Sourcemap upload command requires a single app/service version to associate the sourcemaps with, regular “upload sourcemaps after you build your app so DataDog can display the human readable stack traces” is not applicable to our situation.

I’ve already went through some of the open issues under the DataDog RUM SDK GitHub repo (such as this, this and this) I got the sense that the true MFE support (both for this case and where multiple “init” calls to the RUM SDK is needed) won’t be happening anytime soon.

Solution

Since we don’t want to wait that long to have human readable errors on DataDog and preferably don’t want to maintain a really complex ad-hoc CI/CD solution where we would try to sync the associated sourcemaps and MFE versions across the organization, I have another short term solution that might be maybe done by the DataDog team.

In modern web applications almost all the asset files we deal with is being produced by a build process and the name of the files are being determined by their content. If you ignore hash collision probability; essentially we can say that each file is unique by their name.

If a consumer is in this situation; there can be new opt-in things they can do for uploading sourcemaps, where instead of giving a release version to the upload command, they can just pass an indicator that tells DataDog that these sourcemaps they upload are “global” sourcemaps for the given application. So when DataDog encounters an error that is arising from that file, on top of just checking the sourcemaps associated with that release, it can also check the global one if there’s any match.

It would look something like this:

datadog-ci sourcemaps upload ./build --service my-service --minified-path-prefix https://static.datadog.com --global-release

This would not be a breaking change for rest of the DataDog users. And this is completely up to consumer developers to opt-in. If they can't guarantee deterministic and reproducible builds or if they have files where the file name is always the same but the content might be changing between releases; then they don't need to use this approach (at least for those files).

I assume you put sourcemaps to a bucket and use the app name & version values as folder names in the bucket when it comes to organizing and finding things. For a given app folder there would be a special __GLOBAL__ subfolder and that would be also checked when DataDog attempts to unminify the code for a given error.

Additional context

I know that this won’t be covering use-cases where we really want multiple “DataDog applications” or “DataDog RUM SDK init calls” happening at the same time in a single browsing context. But since what I am suggesting would be an opt-in thing and it would be the consumer app’s responsibility to do this at their own-risk I see this as a “good enough” short term solution for a “partial” MFE support (Similar to this)

As a complimentary thing; DataDog CI CLI can also accept an additional optional parameter to prevent "overwriting" the same file/release version combination and error out (not sure if it already does this or not).

Also see: https://github.com/DataDog/datadog-ci/issues/176

Command

sourcemaps

StanBeckers commented 9 months ago

Hi everyone :) I'm part of Datadog's RUM team, and in order to better understand the needs around Microfrontend support, we created a survey so we can gather your feedback. If you are concerned by this topic and want to help us create a better experience for you could you take 5 minutes and fill it out? The link is below: https://app.ballparkhq.com/record/192df656-cf60-41f1-92d0-83d67d2eb694 Thank you !!

kstieverjc commented 6 months ago

@StanBeckers what was the outcome of the survey and research on this proposed feature?

StanBeckers commented 5 months ago

Hi @kstieverjc !

Thanks for the follow up :)

The feedback was extremely useful, we are currently in the technical exploration phase and are considering a few options. If you want to help us further, you can help us rank these options by completing the following survey: https://www.userinterviews.com/projects/WjBK2WmPlg/apply

Due to the nature of these technical explorations we ask participants to this survey to fill out an NDA, but the first 30 to complete it get $10 as a token of our gratitude

Looking forward to hearing from you !!

jame-earnin commented 1 month ago

Any updates?

nhientrandatadog commented 1 month ago

We are currently working on improving our support for MFEs based on the survey that we shared in February, and hope to have more to share soon. Stay tuned!

jame-earnin commented 1 week ago

@nhientrandatadog Any updates?

DataDog / datadog-ci