artsy / README

:wave: - The documentation for being an Artsy Engineer
Creative Commons Attribution 4.0 International
1.1k stars 121 forks source link

RFC: Trial Expo in Energy and/or Palette-Mobile, Spike on Eigen Risks, Rewards and Effort #543

Open MounirDhahri opened 2 weeks ago

MounirDhahri commented 2 weeks ago

What is Expo? Expo is a production-grade React Native Framework. Expo provides developer tooling that makes developing, reviewing and deploying apps easier. See https://expo.dev/

Proposal

The goal from the RFC is to Incrementally adopt Expo in our mobile Apps:

Reasoning

1. Simplified Update Process

Bare React Native updates are typically complex, requiring manual intervention and can take multiple days to multiple weeks per update.

Video

**Command outcome** https://github.com/user-attachments/assets/0c0733cb-5126-47dd-9c31-a324223626c1 --- **Git diff** https://github.com/user-attachments/assets/ff59e7b5-9f2f-4cf4-89b1-aa83dc970527 Typescript might break after these updates, maybe there will be even some mismatching packages. But fixing ts issues or a couple of mismatching libraries is a LOT less work than fixing +5 packages - especially if you don't know where to look for

2. New Architecture Migration Path

3. Managed Services Benefits

Video

https://github.com/user-attachments/assets/b16255aa-09af-4b0f-b550-7c22b9071871 **Note:** Obviously, the build failed in the end because I didn't set up the env variables but I hope you got my point already

4. Developer Experience Improvements

5. Native Module Development Support

Group 27

6. Industry Standard Adoption

Most of the above is for engineers: What does it mean for users?

Exceptions

No Exceptions

Additional Context

What are the different Expo solutions? I suggest you check out this great video from Expo speaking of different solutions they provide.

Other questions

What would the new flow be like for developers

Example with EAS

![core-development-loop-light](https://github.com/user-attachments/assets/bb17c764-2db4-4d3a-a5ec-43a301200905)

What if Expo goes out of business? Expo has built a strong, active community and is open-source, meaning the core functionality of the platform would continue to be accessible and could be maintained by the community if necessary. Additionally, its popularity among developers and endorsements from established companies make it more likely to remain sustainable. In a worst-case scenario, we would still be able to eject from Expo and return to a bare React Native setup.

Is Expo paid? Expo itself is free to use, with the option to pay for additional premium services (e.g.EAS). The paid services can streamline certain workflows but are not mandatory and can run locally (with limitations).

How much effort would this take? The migration in Energy took around a full working week. I expect the adoption in Eigen to take longer. Probably 2-3 months. I believe it's worth it though given the amount of time we would be upgrading react-native versions, dealing with mismatching versions, onboarding engineers and if we adopt EAS later, setting up the build environment.

What are the next steps?

Why add a different layer on top of what we already have? Expo acts as an abstraction layer that simplifies React Native development, similar to what frameworks like Next.js do for React. This “layer” reduces boilerplate and streamlines workflows without sacrificing the flexibility to create custom native functionality when needed. By default, Expo enables faster iteration and simplifies access to advanced services.

By defaulting to Expo modules, we limit the libraries we can use and loose on the libraries that do not support Expo? That's a great question. A few years ago, the answer would have been yes. But now, the answer is, most of the times, no.

No native code or configuration side-effects: Libraries without native code or configuration side-effects, such as react-native-blurhash, can seamlessly integrate with npx expo prebuild. They can rely on Node Module Resolution and Expo Autolinking to link into the native project without requiring any additional configuration. https://docs.expo.dev/workflow/prebuild/

What does this mean?

Will we need to migrate our native modules? For Energy, yes. I am working on a draft to show how that might look like

Any public experiences of migrating to Expo from companies with big teams?

Any smaller steps that we can take instead of making the full migration? We can first start the migration from RN CLI to Expo CLI and plan the packages and modules migration accordingly.

Resolution

damassi commented 2 weeks ago

Very much 👍 👍 to this.

I've been watching Expo grow and develop over the past few years and while our experience with Next.js should give us some pause, I think the differences are significant enough to keep that comparison separate. In many respects, RN is all about the tooling required to work with what is in effect a client-side-rendered React app, with hooks out to native functionality. Most of our complexity comes from managing versioning, and running a development environment. As documented in the RFC, this is a full-time-job affair for an app that is largely screens interacting with Metaphysics, with minimal interactivity, and not too many native features. The difficulty I think to a large degree has been normalized through years of pain 😅.

Couple things:

Additionally, this RFC only speaks to our current challenges and points at solutions. It says nothing about all of the other abundant features that Expo provides that once we're migrated we'll be able to eagerly explore, and I think the excitement there would be worth it (see: optional file-system-based router for an example of real simplification, being the most obvious).

brainbicycle commented 1 week ago

Thanks for the extensive write up! Looking into some of the benefits of expo I am pretty excited by some of them and this got me looking at their DX and thinking about what is possible here and finding tools I would have not have otherwise heard of. That said as it stands I am opposed to this, at least in the sense of setting an explicit goal of moving all our apps to expo. I am on board with trialing but with the goal of identifying which DX or UX improvements we get and looking at ways to incrementally adopt them through expo or otherwise. I would also prefer to trial in palette-mobile rather than energy. It is my opinion that migrating all our apps to expo is too risky and time consuming for the benefits proposed. I do realize the RFC has been updated to trial in a specific app, just to be clear I am just talking about what the goal of that trial should be IMO, as such I am gonna talk about Eigen because I think it is the reason for my doubts about the goal.

Why Palette Mobile over Energy

The reason I would prefer palette mobile over energy is we set as an explicit goal when creating energy of keeping it as close to the eigen stack as possible to simplify upgrades. If we adopt expo there now we have a divergent stack and need to both handle expo upgrades and standard react native upgrades. Palette mobile we still have this problem but I believe to a lesser extent because palette-mobile app is just a shell for our components library, so it has zero custom native code and minimal dependencies and is likely to stay that way. That will hopefully make upgrades straight forward. Also devs are more regularly contributing to palette mobile so hopefully will get more of a chance to interact with it. If it simplifies the workflow enough or we use EAS maybe we can even get designers running it regularly.

Fears and doubts for adopting Expo

Not convinced by upgrades

That might sound weird, I have spent a lot of time on react native upgrades and would happily never spend any more time on react native upgrades. That said in the next week we will be at 0.75, one away from the latest and in my opinion most of the pain from upgrading has already been paid. If we can continue to stay close to the latest and continue to simplify our app infrastructure this pain will continue to diminish (with the exception of the new architecture which is going to be painful no matter what). This pain is also generally restricted to 1 to 2 devs. Also in order for us to stay on the managed workflow for expo we need to complete a long complex migration that looks to me to be much more complicated than any react native upgrade we have ever done. React native upgrades are also the devil we know, I am not convinced we are not just going to be trading them for problems with expo, which I at least do not know.

Complexity of staying on the managed workflow

I don't know all the steps we need to take to keep eigen on the managed workflow for expo, but at first glance it looks tough. Correct me if I am wrong about any of these but: We have to get all our core dependencies to sync with expo compatible versions: "react-native, react, @react-native/async-storage, react-native-gesture-handler, react-native-reanimated, react-native-screens, react-native-svg, react-native-safe-area-context, react-native-webview" we have to migrate all our non-expo compatible dependencies to expo compatible dependencies, things like @rnmapbox/maps, react-native-image-crop-picker,  react-native-haptic-feedback, react-native-view-shot, react-native-fast-image, we have to replace custom forks with expo compatible modules or more likely make our custom forks compatible with expo, e.g react-native-config, react-native-context-menu-view, we have to migrate our auth code to use expo deps. We have to migrate our native modules to expos patterns, We have to migrate our view managers to expos patterns. We have migrate or get rid of the rest of our native code, we actually still have quite a lot believe it or not and it is not so easy to disentangle, I have tried multiple times. We have to migrate ci and cert management if we use EAS, migrate our Podfile, which is complicated, to expos patterns. The other option of course is to leave the managed workflow which may be viable but then we lose a lot of the benefits.

Old vs New App and restrictions to our path forward

If we were starting fresh today and creating a new react native app without any historical baggage I think it would make a lot of sense to use a framework and suite of tools like expo. As it stands we have a ~10 year old react native app that has a accumulated a ton (too many) dependencies and still has quite a bit of native code and modules. Instead of the case where a new app we get the immediate benefit of many decisions being made for us we instead need to retrofit all the decisions we have accumulated into expos way of doing things. Everybody in mobile has been working hard to simplify and standardize over the years and slowly but surely things are getting better. We have more work to do, reduce our dependencies and get off deprecated dependencies, reduce our native code, replace native experiences with cross platform native experiences, continue to standardize our navigation infrastructure, address performance concerns throughout the app, fix the large backlog of bugs we have, make palette mobile as robust and complete as palette web, implement cacheable throughout the app, get to supporting the new architecture to name just some. I would choose any of these over the migration to expo as priorities for us and we could make a big dent in them in 2-3 months. The other great thing is that many of these things will likely make it easier to migrate to expo should we choose to in the future, however I think making that decision now before we do those things would only restrict us into choosing the expo sanctioned way at every turn.

Quibbles

UX vs DX

I want to push back on replacing a natively built live auctions experience with a webview because it would allow us to get rid of native code. The same would have to happen with city guide. This feels to me like prioritizing dev experience over user experience which I am in general opposed to without a very good justification and even then it needs to be a product discussion as much as a dev discussion. The right path forward is to replace that native code with as good an experience that is more easily maintainable or make the explicit product decision to stop supporting it.

New Architecture is gonna be hard no matter what

Expo might give us tooling to identify which libraries are compatible with the new architecture, but that IMO is not the hard part, the hard part is migrating all of the libraries we are on that are not compatible, and we have to do that either way if we adopt expo or not.

List of companies is a bit misleading

https://evanbacon.dev/blog/expo-2024 This list shows any companies using any expo open source, that includes react-navigation, so we are actually on this list. You can see the triangle icon represents expo, and the other represents react-navigation. That said there are still major companies on this list and I don't doubt that it can work for major apps so this is just a minor quibble.

The Exciting bits for me

Enough negativity!

Reducing Local Build Complexity and Cloud based 1 click builds

This in my opinion is the most compelling benefit shared here. It has been a near constant source of frustration for engineers coming to Eigen. I do believe it is possible for us to adopt EAS without adopting the rest of expo by creating a custom dev client. We still would need to see about pricing and trial it out before making a decision but trialing that or something similar I am definitely intrigued by. There are also some other similar open source options we could explore: https://github.com/shopify/tophat

OTA Updates and Codepush Replacement

Similarly, we need a solution for OTA updates, I know of EAS and Codepush Standalone, EAS looks very compelling for the DX it supports and specifically calls out use cases like testing branches and prs which is something we wanted but couldn't implement with codepush.

THE END

Sorry for the wall and thanks for opening up the discussion. I would personally be open to trialling in palette-mobile with goal of identifying specific benefits and seeing if we want to incrementally adopt some tooling for those specific benefits, in addition EAS is very intriguing and would be open to spiking on that specific benefit and what adoption would take.

araujobarret commented 1 week ago

Thanks for the exploration and for bringing up this idea, I confess I hated Expo in the past when the RN version was 0.47.x, so I can't say much now besides that looks stable and has nice features. From what you've noted and looking at Expo I see:

Pros

Cons

Famous last words

As @brainbicycle mentioned I think it might be good to keep experimenting with it, Energy then Palette but now also involve more engineers in the Palette work so we can collect more feedback about the process. If everything seems positive I'd love to work with Expo but I strongly believe we need to lower our expectations around new architecture and upgrades and keep in mind the extra rework we will have in infrastructure.

MounirDhahri commented 1 week ago

Thanks for the time you took to answer with so much details @brainbicycle . I will try to speak more of few things you raised in your comment to make sure we are aligned on the effort, implications and concerns to come up with the best decision here.

Complexity of staying on the managed workflow

We have to get all our core dependencies to sync with expo compatible versions: "react-native, react, @react-native/async-storage, react-native-gesture-handler, react-native-reanimated, react-native-screens, react-native-svg, react-native-safe-area-context, react-native-webview"

Yes, this is accurate. But it's not something we will need to do. This is something that will be handled for us by using --fix. In Energy, none of the packages above caused issues because expo upgrades will get you to versions that work well together.

we have to migrate all our non-expo compatible dependencies to expo compatible dependencies, things like @rnmapbox/maps, react-native-image-crop-picker,  react-native-haptic-feedback, react-native-view-shot, react-native-fast-image,

I don't think we would need to do that. For all the examples you mentioned above, we either don't need to migrate them because they make no configuration changes and would work out of the box with Expo prebuild or in the case of mapbox and rn-image-crop-picker, there are expo plugins making the switch seemless. For most of the expo packages that I tried migrating, the API is quite the same as popular packages and migrating was neither time-consuming nor complicated.

we have to replace custom forks with expo compatible modules or more likely make our custom forks compatible with expo, e.g react-native-config, react-native-context-menu-view

Eigen currently has two forks, one is for react-native-context-menu-view, that can be kept as it is and we don't need to migrate out of it since it can be automatically linked. The other [Follow Conversation 👉 https://artsy.slack.com/archives/C02BAQ5K7/p1731320188134129?thread_ts=1730988026.713129&cid=C02BAQ5K7]

we have to migrate our auth code to use expo deps. We have to migrate our native modules to expos patterns, We have to migrate our view managers to expos patterns

This might sound like a lot of effort but I don't believe it will be.

 We have to migrate ci and cert management if we use EAS, migrate our Podfile, which is complicated, to expos patterns. The other option of course is to leave the managed workflow which may be viable but then we lose a lot of the benefits.

We have more work to do, reduce our dependencies and get off deprecated dependencies, reduce our native code, replace native experiences with cross platform native experiences, continue to standardize our navigation infrastructure, address performance concerns throughout the app, fix the large backlog of bugs we have, make palette mobile as robust and complete as palette web, implement cacheable throughout the app, get to supporting the new architecture to name just some. I would choose any of these over the migration to expo as priorities for us and we could make a big dent in them in 2-3 months. The other great thing is that many of these things will likely make it easier to migrate to expo should we choose to in the future, however I think making that decision now before we do those things would only restrict us into choosing the expo sanctioned way at every turn.

I agree with all the above, however, I don't think the Expo migration would interfere. If anything, it should be kind of a compass to guide us towards a robust infra over the long term. Given our size, I see the migration towards Expo here as our north star goal and there will be many steps to reach it. We can't for example get to Expo without the navigation work, replacing native experiences etc... so we will need to make those changes. But there are future decisions we would need to make related for example to adopting the new architecture, that Expo can help a lot us with. (for example choosing an expo compatible package, without using expo)

UX vs DX: Migrating Live auction and City guide to RN I want to push back on replacing a natively built live auctions experience with a webview because it would allow us to get rid of native code. The same would have to happen with city guide.

That's a fair point. We currently offer the Webview experience already in Android but I am not sure how well it's performing. In any case, we can still keep the screen as it is and just export it as a native module or rewrite it in RN. Auctions are a big part of our product and I think they could use some UI/UX love.

List of companies is a bit misleading... That said there are still major companies on this list and I don't doubt that it can work for major apps so this is just a minor quibble.

Pardon, that wasn't my intention. I should have paid more attention, I was misled by the tweet https://x.com/Baconbrix/status/1743289259837010023.

Why Palette Mobile over Energy If we adopt expo there now we have a divergent stack and need to both handle expo upgrades and standard react native upgrades

This is actually my goal from this RFC. I want us to evaluate the effort, what would it be like to upgrade the expo sdk. Would it be as hard as it used to be? would it be as fast as promised? What is it that could go wrong?... The knowledge we get will guide with our decision later to adopt Expo in Eigen or not.

Migrating Palette-mobile imo won't yield much benefits given as you mentioned, has no native code, barely any packages and no build pipelines related. However Energy has a little bit of all of these and would help us make a better educated decision in the migration

damassi commented 1 week ago

What I'm reading from @brainbicycle's response is a lot of valid points, but also a lot of fears! These types of fears can typically be evaluated via spikes, in order to make the unknowns more known. We should lean into that, in the same way that @MounirDhahri has already (successfully!) spiked on migrating Energy to Expo. A lot was learned here. We should lean in on this, while keeping the aim for this RFC specific to Energy and Palette Mobile, being simpler apps.

The reason I would prefer palette mobile over energy is we set as an explicit goal when creating energy of keeping it as close to the eigen stack as possible

I'm opposed to this for two reasons:

As Mounirs migration spike has illustrated, the ease in which we were able to migrate speaks very much to the differences between Eigen and Energy. While its ideal that both maintain the same stack, we shouldn't require that in a ideological sort of way. Additionally, Energy has always been regarded as a test bed. If we lean into this, we can innovate, experiment, back out of experiments and more. Vs creating walls.

This feels to me like prioritizing dev experience over user experience which I am in general opposed to without a very good justification and even then it needs to be a product discussion as much as a dev discussion

As someone who has been on the front-line of this for nearly 8 years on web, I've got a lot of first-hand experience about how this is incorrect, and have seen numerous waves of this truth play out over the years with exacting precision. If the DX is good, people will build. If the DX is good, people will rebuild -- and more -- because the greatest bottleneck in feature development is friction enabling the writing/deploying of code, not UX code itself.

When I first joined, building new pages on artsy was regarded as extremely risky. The dev experience was broken, and the time it took to make changes was excessive. The explicit aim of the first Frontend Engineering Taskforce was pointed exactly at this fact. How can we make the DX so good that it unlocks productivity and pages can be built with confidence again? In addition to spinning up new UX patterns, we made the development experience totally transparent, so that developers no longer had to think about it. With this in place (it was very hard, worthwhile work!) what we saw was a proliferation of features and productivity. Looking at Force's Apps folder, we now have 64 applications. Rarely do engineers have questions about how to build, and issues with Force on the tooling side pretty much never come up.

Obviously, web and mobile are very different platforms and the stakes are much lower on Web. But what the FE Taskforce identified at the very beginning were the points of friction, and then we tackled them. Expo, similarly, has meticulously identified the points of friction in building a react native app. So much of this is tooling related -- which intersects DX, not only for those on Sapphire, but for all of us working on the app, things which constantly come up and discourage otherwise good product engineers from contributing fully. Eigen has improved, yes, but the complexity level for our fairly simple UX is astronomically high. Is all of this necessary complexity, or no? Many of the points that @brainbicycle raised are answered by @MounirDhahri, with the end result being -- at least in Energy -- less complexity overall, better DX, and if our experience on Web says anything at all, faster turn-around for our product, while freeing up dramatic amounts of time for our mobile platform team to push the app forward in ways that aren't simply maintenance.

brainbicycle commented 1 week ago

Additionally, Energy has always been regarded as a test bed. If we lean into this, we can innovate, experiment, back out of experiments and more. Vs creating walls.

I don't want to create walls against experimentation, that said I think we should be explicit about what the potential cost is, right now when we upgrade react native or a major dep like reanimated or palette mobile we need to keep those in sync and compatible across 3 repos and we have seen in the past even minor deviations in stack and dependencies can cause headaches. I am pretty confident that if we end up trialing in energy this will cause pain around this process. It is an unknown unknown but I could foresee for example expo only supporting a specific version of a library and us requiring to keep backwards compatibility across versions because of this restriction. Worth calling out past experiments had to be unwound in energy like multiple state management solutions, using an expo provided dep that brought in too much baggage etc. That said I don't think this should prevent us from experimenting but we should be upfront about the cost and trade offs here.

I do hear the argument that palette mobile is not as compelling a test bed but I actually think if we want to learn about how hard eigen is going to be to migrate the best thing to do would be to try to migrate eigen in a separate branch. If we want to see the DX benefits of expo palette mobile I think is better. That said I am okay with trialing it in energy if we think we will learn a lot from it but I want to make sure we are going in with open eyes and have a plan if we need to back out.

Also one of my problems is I don't think an improved DX in energy is high enough of a bar to say we should definitely adopt Expo in eigen. I am actually very confident we will get an improved DX in energy. But 1) eigen is much more complicated due to historical baggage so I don't think the ease of migrating Energy will be indicative of how hard eigen will be 2) the pain I fear from adopting an opinionated framework often comes later, next.js is a good example, I wasn't involved but my understanding is we backed out because the framework started making opinionated decisions we disagreed with and didn't work for our use case, we don't have bandwidth to maintain our own fork of expo should things go that way and we will then be forced to undo any of the changes we did to get here or be stuck on unmaintained libraries. Next.js was trendy popular and gaining steam and still this happened. https://artsy.github.io/blog/2024/03/07/nextjs-at-artsy-retrospective/

But what the FE Taskforce identified at the very beginning were the points of friction, and then we tackled them

I love this approach, I would much prefer we take expo out of the equation, or at least not restrict ourselves to it, and try to identify specifically what these are and see what solutions are available to address them expo or otherwise. For example 2 of the main use cases I see here:

I already wrote why I am not convinced about upgrades, I think pain around local tooling is extremely compelling though and affects every engineer who contributes or tries to contribute to eigen. There are options for us that do not require us to go all in on expo here we could trial. E.g EAS with bare workflow and a custom dev build or something like shopify's tophat lib.

Also I am 100% in agreement that improved DX leads to improved UX and enabling more contributions is better, I am just not in agreement on how to get there.

damassi commented 1 week ago

I mentioned up above about Next.js and Expo, but wanted to make clear that the comparison isn't quite accurate, and want to reiterate that here. Why? Next is a web framework. Expo is an application framework. A web framework includes (in addition to tooling) opinions about how the server-side should function, and how the client should interface with that layer.

Expo brings some restrictions here, but what a RN app is is a client side rendered app -- in effect, create-react-app -- plus a bridge to native code. So the comparison doesn't quite hold. (Its also more complicated 😆)

If we were to look at Next as merely a tooling / dependency management layer, we would have had no problems there. Its defaults worked fantastic. It was everything else that caused problems, and in particular its opinionated manner of writing web apps, RSC, and its alpha-level quality around DX concerns that were previously solved, in Next 12.

jonallured commented 1 week ago

I don't have a lot to add here but wanted to 👏 applaud 👏 the way yall are working through this debate and finding ways to poke at the ideas - really good work!!

As somebody that's not fully plugged into this ecosystem but also has dipped into things over the years I thought I'd just boost a couple things that caught my eye:

Looking to what the React Native teams says can kinda feel like an appeal to authority but also DOES fill me with more confidence.

The fact that Brian has a list of things he'd prioritize ahead of this seems like it's worth listening to and seeing if/how they could lead to Expo could be a good middle ground. 🤷

gkartalis commented 1 week ago

Thanks for this impressive exploratory work first of all and looking into making the dev experience better for everyone!

I will try to summarise my thoughts under each header of your points for all projects and then try to summarise everything in the end.

Reasoning:

1. Simplified Update Process

I have done multiple updates through the years and I do know the struggles of a react native bump. The thing is that though we are already on a state where we are just one major version behind the latest.

Expo --fix updates are impressive I won't disagree on that but at the same time they lock you to specific dependency versions which is not great IMO.

  • Before:
  • In this PR and this PR , we needed respectively 43 commits and 23 commits to upgrade 3 react-native versions. The first PR took 3 days to be merged and the second took 8 days.
  • After:
    • In this PR and this PR, I tried to do the same and upgrade the react-native 3 versions. Both PRs combined required 5 commits, two for the upgrade, and 3 for fixing typescript issues. Both took less than an hour!

This is pretty impressive but, the energy respective react native upgrades are way easier than eigen all the time due to upgrading always eigen first and with our learnings from that upgrade they in the end take way less time than eigen. I also don't consider the number of commits a big factor.

Also another fact that we need to take into account is that when doing a big react-native bump we do wait usually the release to get out so many times we had to wait for that in order to not create overhead on release captain engineers when cherry-picking release fixes.

Updates often break native dependencies, requiring extensive testing and fixes.

I don't see how expo solves that honestly, we do need to extensively test out the app after each upgrade either way.

2. New Architecture Migration Path

This one is pretty impressive but still the react native 76 with react native architecture enabled by default is still in beta for expo so far, but the tool is really useful though.

3. Managed Services Benefits

OTA - That is a different topic completely since it is completely separate from expo and I do love the idea of migrating Codepush to the expo solution but it is completely decoupled from expo and we can introduce it without adopting everything.

Callstack did a really interesting podcast around this topic - https://www.youtube.com/watch?v=X7EMrgI3A9g


> Build services eliminating the need for local build tooling

Would we really want to adopt this as well? This is a paid service as far as I know

4. Developer Experience Improvements

I do like most of the dev experience that expo offers but it also comes with some caveats:

5. Native Module Development Support

We do have a bunch of complicated native modules in both eigen and energy, I would be really interested to see the diff i.e. in energy of how we migrated the Albums from iOS to expo native modules.

6. Industry Standard Adoption

Coinbase Shopify Ubisoft Tesla BlueSky-social (also open source)

  • Active community support and regular updates to packages
    • Since everyone is using the same packages from expo, everyone is focused making sure they're up stable and receive regular updates. 
 I think I disagree on this one, not sure if they do publish regular updates for their expo modules. An example is that bluesky-social that uses expo has much more patches than we do have and most of them are around expo modules.

(They are also a bit behind when it comes to the latest react-native version and expo they have expo v51 and react-native 74 since expo v52 and 76 is not ready yet)

Most of the above is for engineers: What does it mean for users?

  • Smaller bundle size with Tree shaking, is the biggest selling point of expo for me right now but they are some custom alternatives that do work with metro bundler. for example https://www.npmjs.com/package/@rnx-kit/cli

To summarize:

If we do have solutions for the questions above and adopting expo is quick / and won't break many things /happy to try it out in either palette-mobile or energy.

I do believe slightly though that palette-mobile would be a better candidate to understand if it does work mostly due to more devs having to work with it, energy is also a potential candidate but how do we evaluate it since we don't work that often there?

damassi commented 1 week ago

Regarding palette mobile - I'm curious what migrating looks like so I'm going to timebox spiking on that over there today.

brainbicycle commented 1 week ago

After discussions and reading peoples thoughts on this I just want to clarify I am not opposed to trialing, POCs and information gathering and do think those are worthwhile and definitely necessary to make a call here.

I think the title of the RFC is throwing me a bit, putting a thumbs up on "RFC: Migrate Mobile Apps from bare React-Native to Expo" I don't think I am ready for, but the actual text of the RFC and more measured approach of trialing in Energy, Palette-Mobile to gather more info I am supportive of. It will have some downsides just by virtue of having some apps on bare react native and some on expo but I think that is okay and we can deal with. I wonder if we would be open to changing the RFC to "RFC: Trial Expo in Energy and/or Palette-Mobile, Spike on Eigen Risks, Rewards and Effort" or something similar.

Even if we trial and decide not to go forward I think it will be worthwhile to see what is possible with DX and what we can improve.

I think I have voiced my concerns with eigen but it would be very helpful to gather more information. Some specific things I personally would like to see from said trials / spikes:

  1. How hard will it be and what exactly do we have to do to migrate Eigen?
  2. What kind of restrictions does it place on us going forward?
  3. What are the risks if expo starts making decisions we disagree with and we have already migrated? (this scares me the most to be honest and I think it is the hardest to predict ahead of time)
  4. What are the specific benefits expo brings, are there other ways or piecemeal ways to get them to de-risk us from going all in?

Number 3, I think is the most difficult in my mind should we decide to go all in, open to suggestions on how to alleviate that concern, maybe: waiting a while through multiple upgrades and getting a better understanding of issues we run into in energy and or palette-mobile, looking at people in the community if anyone who has adopted expo and then backed out and why.

Also RE: palette-mobile v Energy, maybe the answer is trial in both? I think having one app on expo is not going to be much different friction wise than having 2.

MounirDhahri commented 1 week ago

RFC: Trial Expo in Energy and/or Palette-Mobile, Spike on Eigen Risks, Rewards and Effort

This is a great one - I will update it.

damassi commented 1 week ago

Just stumbled on this, which is an interesting read given their 90 member dev team.

They kept their fastlane / custom CI config, and in terms of native code, ended up

carefully crafting around 30 in-house Expo config plugins.

Improvements to DX track:

The transition to Expo has been a game-changer for our mobile development process. The feedback from our engineers has been overwhelmingly positive, with many citing the improved developer experience as a major benefit.

Also onboarding new engineers on mobile development is now significantly easier as it only requires 2 commands:

install the expo-dev-client run the JavaScript bundler

And the time spent (no longer) building is significant. They break down the numbers and come out with this final point:

We currently save our engineering team around 30 hours per month of building time

Their scale is much larger than us, but in any case, worth a read 👍

joeyAghion commented 1 week ago

In tech radar terms, it sounds like we've "assessed" and are interested in a "trial." Importantly, (1) we might incur extra maintenance or cognitive load during the trial, and (2) may still decide not to adopt Expo after getting that production experience and have to undo or redo some work. Those are OK, and just the cost of continually evolving our stack.

I'm an Eigen outsider but find this opportunity compelling for a few reasons:

damassi commented 1 week ago

Heading out for a few days of OOO, but wanted to post my progress on palette-mobile: https://github.com/artsy/palette-mobile/pull/280

Coming along very nicely 😎

brainbicycle commented 1 week ago
  • In our Rails projects, we've found the opinionated framework to ease development not necessarily by being the "best" opinion all the time, but simply by removing ambiguity and ensuring bugs surface early+elsewhere. That can extend beyond DX to production quality: the less novel the stack, the more robust the output.

Just to clarify, I am not against frameworks as a concept, nor do I think they have to be perfect to be very useful. And I very much agree with "the less novel the stack, the more robust the output." Regardless of outcome of this one of our goals is and should be to get eigen to as standard and boring get out of the way stack as possible. As Chris say's web frameworks and mobile frameworks are not 1:1 comparison but when I think about evaluating evaluating the risk of adopting a framework some of the metrics I would consider are overall popularity (just in raw numbers how many people are using it in production), age (how long the framework has been around?), stability of the foundation (what is this standing on top of and how likely is it to move in unpredictable or unsustainable ways?).

Rails for a ruby web project is a no brainer by these metrics, it is use by 100s of thousands, probably millions of people, it has been around 20 years and it is built on top of the open web and tech that is open.

Expo on the other hand is not so clear cut to me, the blog https://evanbacon.dev/blog/expo-2024 identified ~2k but included apps using react navigation so it is probably closer to a ~1k or the high hundreds, expo has been building libs for react native since 2017 but the form it is today only started in 2019: https://medium.com/the-exponent-log/expokit-2019-1e5cb02106f8, it is built on top of react native which is open but not entirely stable (api wise) which in turn is built on top of mobile ecosystems which are closed and also not stable api wise. Them being the recommended path for new apps by react native should be a big boost in regards to popularity but that is a very recent development. Also it being mostly a tooling layer may be a big advantage here but I think we need more info and some time.

Not looking to re-litigate and like I said very open and supportive to POCs and information gathering, (we should do it!) but I do want to at least make clear why I think there is risk here and we should be careful before going all in.

damassi commented 1 week ago

Had a pertinent thought when migrating Palette yesterday, thinking about some past work with dependency management while on MoPlat.

The fact that Expo locks (popular) library versions to certain blessed numbers I think is wise, because how much time have we spent trying to upgrade, say, reanimated, only to realize that something is out of sync with a sub dependency, or some other thing is incompatable? By establishing a pre-build environment with known good defaults we're ensuring that configurations of things are sure to work together, and this is reassuring.

For certain types of apps I can see this being somewhat unappealing, but for an app like Artsy this (feels) like a win. The constraints are here are freeing, even if it might mean that we need to wait a bit longer to upgrade to the freshest new thing, which is often a fraught endeavor in any case.

That said, its hard to say how much friction this sort of behavior will bring. But in theory, I like the reassurance that once deps are aligned with npx expo-doctor, the app will compile and run. In Palette-mobile, we were always within the minor-version range. There's possibly something very liberating to this given the often overwhelming complexity that native version management brings.

MounirDhahri commented 1 week ago

Thanks for spiking on palette-mobile @damassi and for adding instructions for that

By establishing a pre-build environment with known good defaults we're ensuring that configurations of things are sure to work together, and this is reassuring.

I agree with that, we might still have to fix a couple of packages. But in the case of Energy, for 3 RN versions upgrade (2expo SDKs) I only had to fix one package to use a newer version and that was Sentry.

MounirDhahri commented 1 week ago

Not looking to re-litigate and like I said very open and supportive to POCs and information gathering, (we should do it!) but I do want to at least make clear why I think there is risk here and we should be careful before going all in.

Fwiw, I actually already created a POC that drove me to create this RFC. Considering the number of users of Energy, the smaller code-base, and the little features it has. I would suggest instead: rolling out Expo to production in Energy.

That will help us learn a lot about:

Only if our experience is positive, we can decide to gradually migrate Eigen to Expo.

anandaroop commented 6 days ago

Wow, such a comprehensive RFC and discussion 👏🏽

As an Eigen part-timer who has felt this pain repeatedly…

changes to native code, which required rebuilding the project again.

  • If everything goes well […]
  • If something doesn't go well, which usually happens for folks who don't use Eigen much […]

Eliminates complex local environment setup (Android Studio, Xcode, etc.)

…I am very interested in this.

In fact I literally am reading this RFC because I am waiting for a full rebuild of Eigen, and it has failed ~twice~ thrice now. If Expo can streamline this process as advertised, that alone might sell me on it.

Though it sounds like this is based on the EAS paid add-on. Unclear to me how much of this benefit we can reap if we don't go the paid route.

I defer to the mobile experts here, but the reframed goal of trialing in the two simpler apps plus maybe spiking in Eigen sounds sensible to me.

damassi commented 3 days ago

@anandaroop - Something to note from the Palette migration was:

This would be even simpler with EAS, but just wanted to point out that we could keep these build environments around (and perhaps we should), but lean into expos tooling to take an enormous amount of friction out. Set up compilers, align library versions and settings, and good to go.

MounirDhahri commented 3 days ago

but just wanted to point out that we could keep these build environments around (and perhaps we should), but lean into expos tooling to take an enormous amount of friction out. Set up compilers, align library versions and settings, and good to go.

@damassi I agree with that. Thanks to expo prebuild --clean and the built-in artifacts cleaning, we are going to be generating a new clean native code that will be more stable and will get us in a LOT less situations like this - where you need to guess what's exactly broken and run every possible cleaning command. Such situations are what slows engineers the most when working on Eigen. We had at least 4 similar complaints just last week and I think none of them would have been raised with expo prebuild --clean because it's a clean project basically.

About EAS, yes, it will make this even easier but I also don't know think we need to decide on that jump now.