RFC: More Relaxed CodePush Usage for Folio

[damassi]

Status: Adopted

More Relaxed CodePush Usage (For Folio)

We've talked about Microsoft CodePush for a long while, and over the past handful of months have started using it at Artsy -- first in Eigen, and now in Folio. Currently, standard operating procedure is to only use it for critical hot-fixes, and to continue following standard app release best practices; this is fine, but for an app like Folio I would like to argue for a more relaxed approach in order to better feel out CodePush's capabilities, define a more forward-thinking app release strategy, accelerate momentum, and to also surface rough edges.

There are a few reasons for this, primarily focused around Folio's low risk-profile:

• Currently we have very few users on the app (partners, and only some partners)
• It is a "closed" app, requiring login
• It is a new app, and partners have been quickly surfacing issues that they expect fixed and other missing features
• It is small and narrowly-scoped, feature-wise
• It's a great testbed

Once these learnings have been had, perhaps we can start applying them to our Mobile app. First, however, we need to collect intel.

A Real Life Example

Two things came up recently that had some criticality involved, and allowed us to test out CodePush's rapid feature potential.

1) We totally overlooked artworks with multiple images. A gallery reached out asking about a downgrade to the old Folio; the inability to see their full catalogue was a deal-breaker. Deploying this fix from dev to prod was nearly instant.

2) Shortly after Folio was released a VIP gallery reached out complaining that compared to the old Folio, the new one was far less usable as a presentation tool. The artist image avatars didn't invite browsing by collectors, since a collector couldn't see the nature of the artists work from such a small visual footprint. Using CodePush, we were able to add a new setting toggle for large artist images and deploy right away.

In both of the above examples we were able to send things out with two simple commands:

$ ./scripts/codepush/deploy-to-codepush.sh 'Canary' 'feat(artwork): support multiple artwork images'
$ ./scripts/codepush/promote-to-prod.sh 100

The commands run quick. So quick that one might think it didn't even work! But restarting the app, there the feature was. Win.

Risks

While there have been historical fears around using codepush and facing rejections from the app store, a closer look at the fine print should alleviate some concern. From the docs:

Google Play:

Third paragraph of Device and Network Abuse topic describe that updating source code by any method other than Google Play's update mechanism is restricted. But this restriction does not apply to updating javascript bundles.

This restriction does not apply to code that runs in a virtual machine and has limited access to Android APIs (such as JavaScript in a webview or browser).

That fully allows CodePush as it updates just JS bundles and can't update native code part.

iOS App Store

Paragraph 3.3.2, since back in 2015's Apple Developer Program License Agreement fully allowed performing over-the-air updates of JavaScript and assets - and in its latest version (20170605) downloadable here this ruling is even broader:

"Interpreted code may be downloaded to an Application but only so long as such code: (a) does not change the primary purpose of the Application by providing features or functionality that are inconsistent with the intended and advertised purpose of the Application as submitted to the App Store, (b) does not create a store or storefront for other code or applications, and (c) does not bypass signing, sandbox, or other security features of the OS."

CodePush allows you to follow these rules in full compliance so long as the update you push does not significantly deviate your product from its original App Store approved intent.

Additionally, React Native CodePush is a popular library, with over 150k downloads last week alone.

Folio is a great place to test some of this out. And in the worst case scenario, we simply have to remove it. So why not work with this a bit, and see what we can learn. There is potential here beyond hot-fixing.

Release Process

This might be better scoped by Sapphire, but some napkin-math thinking here would be to consider CodePush and App Store releases as two distinct things. CodePush promotions are always flagged by a Production tag, which in effect serves as our change log. Full releases can parse these tags for metadata, serving as a means to periodically update the 'New Features' section in the App Store. But "releases" as commonly understood are secondary. Rather, we start thinking about Folio in terms of Continuous Integration, just like web, and see how things go.

Revising our long-term mental model might feel weird at first, but I suspect it has the potential to supercharge productivity on mobile while simultaneously leading to a much better product, as getting those two critical features in over the past two days illustrates. Feature releases to users become just another routine act of hitting "merge" on a deploy PR -- a process we should setup for CodePush in our mobile repos.

Additionally, @brainbicycle has already worked out safeguards against codepushes against branches that have native code changes included, which fail the push. We can raise this via Danger in the Deploy PR, via CI.

Resolution

Level of Support

Next Steps

Exceptions

[MounirDhahri]

I am in for more ease releasing updates without going through the stores or waiting for the store update. The faster we release such updates, the happier we can keep our galleries. so 👍 for that

When it comes to rethinking the release process in favor of continuous integration in Folio, I think that's going to be a subject we need to be careful about and how often we do it. Some updates might bring some native code changes and can potentially break the app (speaking of palette-mobile deps). I believe such updates will fail thanks to @brainbicycle native code changes check, but when they do, we will need to get back to the normal process of releasing through the store.

Regardless of my worries here, I do though like the premise of it. I love CodePush canary builds in Eigen, they make my life so much easier and QA a ton faster. I would then also say yes to trying this out here, and figuring out a way to make sure we don't release an over-the-air update that breaks the app.

[damassi]

Yes, its crucial to mention here that Brian has already worked out significant safeguards on the native code side! Thank you for raising; I will update the RFC mentioning it.

[brainbicycle]

I am mostly onboard with being a little more aggressive with codepush in folio. With the caveat that I think it would be better to ramp up and see how it goes than to go all out continuous delivery immediately. Also if we did that would require some not insignificant planning and updates to our ci and release processes.

Two risks I am concerned about:

1. Surprise rejections + having to remove codepush: The rfc quotes the codepush docs, but the docs of the tool are always going to try to assuage concerns like this. I am more interested in the guidelines of apple + google and if anyone has faced issues with this before. There are anecdotal rejections in the issues in the codepush repo: https://github.com/microsoft/react-native-code-push/issues/2416 And historically there have been tools like rollout.io that promised similar and eventually got a blanket ban from the App Store. That said they are pretty few and far between for a widely used tool, and the guidelines of apple do seem to allow for this. However the end of the day the review process can be a bit arbitrary and up to the interpretation of individual reviewers so I think we should consider that there is a non-zero possibility that we could end up having to remove codepush should we face that issue.

2. Releasing bad / incompatible updates: @MounirDhahri raised the concern about incompatible updates that requires an update to the native code. We've seen this in practice as just a crash on startup. We have a basic script that checks if native code has been modified but there is a known gap, we don't have locked android native deps and there may be unknown gaps that could lead to an incompatible release. In the short term we could do a manual basic sanity check before releasing any updates into the wild but if we wanted truly continuous integration we need some way of confidently + automatically testing these releases before going out to prod.

For those 2 reasons would be in favor of folio being the vanguard for a more aggressive release process that might lead into more continuous delivery later. And if it goes well bringing it over to Eigen.

At a high level thinking something like this:

1. Any release that can be done via codepush we do via codepush with manual sanity check QAs before rolling out to prod (basically check if the app crashes)
2. When a native change is required do ad hoc standard releases with a standard regression QA
3. Keep track of how it goes, how we might adopt a more automated and continuous process, retro and plan next steps.

[damassi]

Cool, thanks @brainbicycle. Gonna mark this one as accepted and continue pondering some pipeline improvements 👍