Automattic / wp-calypso

The JavaScript and API powered WordPress.com
https://developer.wordpress.com
GNU General Public License v2.0
12.4k stars 1.98k forks source link

Create Woo Express passwordless authentication feature flag #87635

Closed chihsuan closed 6 months ago

chihsuan commented 6 months ago

Issue description

Project thread: pdibGW-2FU-p2

Create a Woo Express passwordless authentication feature flag and add scaffolds so we can work on passwordless tasks without affecting the existing flow.

Acceptance criteria

rjchow commented 6 months ago

@chihsuan Would the cleanest way to do this to be just checking a query param on wp-calypso? That would make it easy to test as well as not having to futz around with feature flags.

We can assign the query param on wccom nux according to the A/B test assignments. For funnel/conversion rate analysis we can either add a flag in the tracks event or add a whole extra event.

chihsuan commented 6 months ago

We can assign the query param on wccom nux according to the A/B test assignments. For funnel/conversion rate analysis we can either add a flag in the tracks event or add a whole extra event.

Since we currently do not support cross-platform anonID, I'm not sure if initializing A/B testing assignments in wccom is appropriate.

@ilyasfoo Since you're the expert. 😄 What do you think?

rjchow commented 6 months ago

If that's not workable then we can use https://github.com/Automattic/wp-calypso/pull/58404/files#diff-86078c0035544c18a00cfd610a7ce7bd4acd811fd06a53167668a1b067e40e43 as inspiration , and perhaps check that experiment assignment && the feature flag toggle via query param

Edit: Looks like feature flag toggle via query param doesn't work in prod: https://github.com/Automattic/wp-calypso/blob/ff3a200efd3cf02040eec9a0b62b4b24dbc609e4/config/README.md#L60

ilyasfoo commented 6 months ago

Since we currently do not support cross-platform anonID, I'm not sure if initializing A/B testing assignments in wccom is appropriate.

Good question, with a/b test, AFAIK we could link users between platforms only if they eventually log in. Any metrics tracked between navigating to WPCOM and logging in will not be reliable.

Edit: To add, they need to be logged in in both WCCOM and WPCOM, since those platforms needs to fire aliasUser individually which will link each platform's anon_id to the user_id. As far as I understand our Woo Express flow will return to WCCOM after logging in, so we should be covered.

ilyasfoo commented 6 months ago

Apologies if I misunderstood, but do we have anything we need to experiment in WCCOM? I.e: will there be a treatment vs control in WCCOM? Otherwise, I think we should be safe to start the a/b experiment in WPCOM 🤔

rjchow commented 6 months ago

It makes the design a bit cleaner and easier to tidy up afterwards rather than having to append a query param after reaching wpcom, or check the experiment assignment at every branch.

So to sum, it should be ok to conditionally append a query param that enables the passwordless flow, at the WCCOM NUX exitpoint

ilyasfoo commented 6 months ago

@rjchow I'm not sure if I understood, would we still use ExPlat for A/B experiment? Or are we going to the tracks manual analysis route

rjchow commented 6 months ago

I have in mind that we'll put in a && experiment assignment === treatment at this point https://github.com/Automattic/woocommerce.com/blob/trunk/themes/woo/start/components/installation/installation.js#L195 , do you think that'll work?

edit: tagging @ilyasfoo 😬

ilyasfoo commented 6 months ago

@rjchow Sorry I missed the ping!

do you think that'll work?

Since the user is anonymous, the tracks fired after user is transferred to WPCOM will not be linked to the user if the user never logs in and returns to WCCOM. In ExPlat, these users will simply seen as dropped-off as soon as they left WCCOM.

rjchow commented 6 months ago

completed by

87887

Automattic/woocommerce.com#19759

chihsuan commented 6 months ago

Just re-open this. We decided to start an experiment in Calypso so we should add a feature flag in Calypso instead and remove the feature flag in wccom

p1710132042112829/1709620856.653379-slack-C01SFMVEYAK

rjchow commented 6 months ago

Just note that feature flags in calypso can't be toggled via query params! https://github.com/Automattic/wp-calypso/issues/87635#issuecomment-1964111761