Support running in browser extension service workers

kdawgwilk commented 2 years ago

Is your feature request related to a problem? Please describe.

We would like to use the LD js-client-sdk in a chrome extension background service worker but the SDK uses window and document APIs which are not available in extension service workers as of manifest v3 such as XMLHttpRequest which was replaced with fetch https://developer.chrome.com/docs/extensions/mv3/migrating_to_service_workers/#workers

The global scope for service worker is significantly more limited and doesn't have many of these features. Most notably, service workers don't have access to the DOM. Workers no longer provide XMLHttpRequest, but instead support the more modern fetch()

Describe the solution you'd like

Have fallback options to ensure the SDK can operate in a more limited environment such as service workers

Describe alternatives you've considered

we have considered poly-filling in the window/document APIs that the library uses to work around the problem

cc @DallinVan

kinyoklion commented 2 years ago

Hello @kdawgwilk,

Thank you for the request.

Filed internally as 162088.

Thanks, Ryan

sghsri commented 2 years ago

@kdawgwilk do you mind telling how you planned on polyfilling the window/document APIs, I'm also trying to get the LD client to work in a service worker and running into some issues

kdawgwilk commented 2 years ago

@sghsri I haven't worked on polyfilling these yet, but there were only 2-3 apis that needed to be polyfilled and they are mostly tied to the document lifecycle so I was planning on tying them to the service worker lifecycle instead

sghsri commented 2 years ago

@kdawgwilk oh ok, that's awesome to hear, thanks for responding! I am working on doing something similar. Are you writing the polyfills yourself, or have you found any packages that already do it. Thats been the main roadblock so far for me

kdawgwilk commented 2 years ago

I haven't been able to find something that already exists yet but I haven't done too much research

kinyoklion commented 2 years ago

Hello @sghsri and @kdawgwilk,

I don't have any progress to share on this at the moment, but I do think it would be worth discussing what would likely be the SDK solution.

The client SDK is layered. There is a js-common-sdk, which theoretically isn't browser specific. This is where the bulk of the implementation is, and it is used in a few different SDK implementations.

The js-client-sdk, this repo, is the browser specific layer. The primary purpose to provide a platform for the common SDK to use.

The js client SDK is really a content SDK.

My initial thought is a js-worker-sdk, which implements a platform which can be used by common. Though some things in the client SDK could just be updated and would gain compatibility. It gives a clear, and more constrained target.

Thanks, Ryan

kdawgwilk commented 2 years ago

That seems like a pretty good approach. There is a lot of functionality in this SDK that would be the same for both, however, so there may be a bit of duplication, but it may be worth the trade-off.

milankonir commented 2 years ago

👋 We have ran into the same issue while migrating extension to MV3 and decided to build an extension background service worker specific SDK on top of the js-common-sdk.

We have a working prototype that:

Uses fetch to perform http requests against LD services
Provides localStorage implementation backed by chrome.storage.local to support the localStorage bootstrap option (important to make flags available as soon as possible when background service worker suspends / wakes up).
Flushes analytic events on extension onSuspend event

We are planning to open source / contribute this back to LD if there's interest.

What would be a tremendous help in maintaining this SDK going forward is if launchdarkly-js-sdk-common provided typings for the initiaize method - for now we had to reverse engineer the information from the launchdarkly-js-client-sdk.

eli-darkly commented 2 years ago

@milankonir: We appreciate that you've put a lot of work into this. However... we would be very wary of making js-sdk-common into a supported API for customers to build on. It is an implementation detail, and treating it as an internal library has allowed us to have the freedom to make internal design changes to the SDKs as necessary, without having to worry that we'd be breaking someone else's project. In fact, it's not unlikely that there will be a complete rewrite of that code in the near future, which unfortunately means the prototype you've built will probably not work with future versions.

milankonir commented 2 years ago

@eli-darkly Thanks for the heads up. I can understand the reasons why you don't want to turn the js-sdk-common into officially supported API for 3rd parties to build on top of.

Do you have any rough timeline on when does the rewrite might happen? The extension manifest version 2 sunsets by the end of the year so we need some solution sooner rather than later. I have a feeling the custom SDK on top of js-sdk-common can be a good intermediate solution for us.

I don't know if the extension scenario is something you want to give long term support for. If not, it would be great to consider making some parts of the SDK extensible during the rewrite by allowing consumer to provide custom implementation for things like networking, storage, host lifecycle...

kinyoklion commented 2 years ago

@milankonir We cannot commit to any dates, but we are going to be starting with the server SDK and then applying those lessons back to the client. We will also likely be able to share a lot more implementation between the client and server SDKs. One of the major aspects we are tackling in the server SDK is to have an abstraction specifically for the platform. This platform abstraction will include things like a fetch abstraction, event streaming abstraction, cryptographic abstraction, etc.

The implementation of which will be in typescript and make it very much more prescriptive as to what will be required to implement a platform. The primary purpose of these will be to allow for internal development and maintenance, and they will be more subject to change than the public API.

I wouldn't depend on a time frame coinciding with the deprecation of MV2 though. We have recognized the same problem though, and also that it will be continue to be a problem as JS runs in more places, and each of those places has their own limitations and sandboxing.

milankonir commented 2 years ago

@kinyoklion Great info, looking forward for the new SDK! Until it lands, I have published an SDK that is compatible with browser extension service worker environment (MV3) for fellow extension developers to use.

Source codes: https://github.com/getoutreach/launchdarkly-js-extension-sdk

Npm: https://www.npmjs.com/package/@outreach/launchdarkly-js-extension-sdk

rzec-allma commented 1 year ago

Any update on the status of this?

kinyoklion commented 1 year ago

@rzec-allma There isn't an update at this time.

lidorkook commented 9 months ago

@rzec-allma There isn't an update at this time.

Hey, do you have an estimate when it will go into development, or is there another official solution for the meantime? Google are strongly recommending completing the migration into MV3 until next June. https://developer.chrome.com/blog/resuming-the-transition-to-mv3#the_phase-out_timeline

aam1r commented 7 months ago

Hi @kinyoklion,

We're currently using the fork that @milankonir shared, but it hasn't been updated in two years.

With the Chrome MV3 deadline looming, we're seeking any updates or guidance on making LaunchDarkly work with Chrome extensions. Your insights would be greatly appreciated.

kinyoklion commented 7 months ago

Hey @aam1r,

We have started moving client-side SDKs into the new structure with a separable platform. We started with React Native, but will be moving on to other client-side SDKs.

I am going to ping @yusinto as he is working on related items. This thread should provide some useful context.

I am not certain if we will specifically have an SDK configuration targeted at extensions or service workers, but we will have greater flexibility with our core SDK to make different derivatives.

Thanks, Ryan

patrickdbakke commented 6 months ago

Hey @kinyoklion - writing on behalf of Superhuman.

Are there any updates on the timeline for an officially supported LD SDK compatible with web workers? As @aam1r mentioned, the chrome manifest v3 deadline is around the corner (a soft launch as soon as June 15).

The fork from @milankonir is great, but is several years out of date. Unfortunately, it doesn't appear to support everything, including missing support for multi contexts.

We're making due with some hacks, but would love to use an officially supported implementation as soon as it's ready.

kinyoklion commented 6 months ago

Hello @patrickdbakke,

There is not any update currently on a timeline. The core of the client-side SDK exists now here: https://github.com/launchdarkly/js-core/tree/main/packages/shared/sdk-client

It shouldn't be doing anything that will not work in a service worker or theoretically an extension. It isn't an SDK by itself though and requires implementations of a platform be provided to it.

So far it has only been used for development of the React Native SDK.

I would recommend that you also speak with support to request this as a feature.

Thank you, Ryan

patrickdbakke commented 5 months ago

Hey @kinyoklion and other folks!

Thanks for steering to the new sdk. We built a web Platform integrating with that new client-side sdk as you recommended!

That's here: https://gist.github.com/patrickdbakke/c0469e67dfda0a5cf3ed3de4129a7d01

We're sharing our implementation here in case other folks or teams can benefit from our work until a time when launchdarkly officially supports a web sdk compatible with manifest v3 / web workers.

Some call outs - There's a few internals in this implementation that are not parameterized enough for a general purpose launchdarkly sdk. Other folks looking to borrow from this implementation would need to update these lines:

the platform and sdk name
using process.env.LAUNCHDARKLY_CLIENT_ID and process.env.VERSION
consider another uuid library - this currently uses our own superhuman-node-uuid

Tangentially - @kinyoklion, Can you clarify the requirements for the Hasher interface? The code description says sha1 is required (here), but your react-native implementation omits sha1 (here). In our testing, it doesn't seem actually needed.

-Patrick (from superhuman)

kinyoklion commented 5 months ago

Hello @patrickdbakke

The code description says sha1 is required (here), but your react-native implementation omits sha1 (here). In our testing, it doesn't seem actually needed.

This is because we only use a subset of the hashing capability for client-side SDKs. Originally I made this interface when implementing the server-side SDKs. I should probably update the comment to indicate which are specifically needed for which paradigms. (I originally thought we would ideally support everything always, but it is less than practical for react native.)

launchdarkly / js-client-sdk

Support running in browser extension service workers #253