Separately Deployable Engines

[richgt]

Looking for thoughts here, and how to proceed. What we'd like to be able to do is deploy our engines separately from our main application. We have a giant monorepo app that we're working on splitting into engines, but even engines are beholden to the main application to be deployed, causing some pain / delays. We'd like to be able to deploy engines separately to their "mount point" when they've been updated, such that the main application could just pick up the new code, instead of having to update a version in a package.json, or something similar.

Please let me know if there's a better / different way to go about suggesting this.

[buschtoens]

We're also extremely interested in this, but so far only did some thought experiments. I guess that it will only really / sustainably be possible, once ember-engines uses embroider under the hood, primarily because of the very involved addon / dependency deduplication mechanism.

Running wild with imagination here and dreaming one step further though considering route-based code splitting, it seems that we don't necessarily have to reserve this capability for engines, but could maybe even offer these incremental / partial deployments for regular apps. We could even pair this up with some sort of hot code reloading, maybe even in production deployments.

[john-griffin]

We have been looking for a similar solution to this also. Maintaining a few large ember apps with many common dependancies could really be improved via something like this.

[phndiaye]

Hi @richgt @buschtoens @john-griffin , I've just seen this question but I can share how we've been doing it for a couple of years at @upfluence :)

Background

Note that our scenario is the reverse of yours as we went from multiple Ember apps, bound together under the same roof at the Reverse Proxy level:

• URL /a loads app-a
• URL /b loads app-b

This allowed us to work only on specific parts of our software and each of them had their release cycle. The main pain points being:

• on our end: bumping common dependencies in all repositories when there are new features
• on the client's end: weird behavior as transitions between parts of the software actually load a whole Ember app (with its own assets everytime)

Also note that any configuration like backend URLs were based off Environment Variables.

The migration to Engines

Because we already had our apps split by concern, we just migrated them to engines and mounted them at the right path in a parent app we created (which is the single app deployed now). Then they were configured just like any Ember addon, just from the parent app's config/environment.

Fun fact: because this took a bit of time and we did not want to stop shipping features/bugfixes, we deployed the dummy apps during the transition period and they worked perfectly without anyway noticing a change. We just had to adapt the rootURL depending on whether it was handled by the parent app or the reverse proxy.

Working on the engines independently

However, like you, we did not want to lose our ability to focus on a single app when coding, so what we did is that we mounted the engine in its dummy app. There is just a minimum boilerplate:

• mounting the engine on dummy/app/router (looks like an app mounting itself 😅 )
• "self-configuration": we basically put the expected addon configuration in the dummy app's config/environment

In short, we can say that each engine is its own parent. And all shared features are brought by addons.

Caveats

Transitions between engines We don't have much of these but what we plan on doing is to create mock routes to transition between engines. We don't need to test the other engines when working on one, just that the transition doesn't raise any issue. LinkToExternal is already doing that work.

Updating the parent app At the beginning we were doing it by hand, which is a bit annoying but we are now relying on our CI to automatically bump the engines versions when they are deployed to the npm registry. Today, the parent app is mainly bot operated and we only work on engines (and shared addons).

(cc @AlexisMontagne)

[wagenet]

I'm closing this due to inactivity. This doesn't mean that the idea presented here is invalid, but that, unfortunately, nobody has taken the effort to spearhead it and bring it to completion. Please feel free to advocate for it if you believe that this is still worth pursuing. Thanks!

[villander]

@wagenet I think we need support from the core team here to move it forward. It's sad to see that every RFC opened for Engines has been inactive because Engines seems abandoned by the core team and the community is drowning in frustration. Close an RFC because it has not been reviewed by the core team, instead making a big call out to them, is another shoot in our heart.

[richgt]

FYI, I have been working with @ef4 and @wycats on this. We expect to have a solution for Polaris.

On Mon, Jul 25, 2022 at 9:52 AM Michael Villander @.***> wrote:

@wagenet https://github.com/wagenet I think we need support from the core team here to move it forward. It's sad to see that every RFC opened for Engines has been inactive because Engines seems abandoned by the core team and the community is drowning in frustration. Close an RFC because it has not been reviewed by the core team, instead making a big call out to them, is another shoot in our heart.

— Reply to this email directly, view it on GitHub https://github.com/emberjs/rfcs/issues/677#issuecomment-1194074696, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJRK6MMJPMJVNWWD5NSLYLVV2LYZANCNFSM4S47GTNA . You are receiving this because you were mentioned.Message ID: @.***>

[wycats]

I'm gonna reopen this to indicate that there's some activity here.

@villander do you think you can sync up with me and @richgt to figure out a path forward? We'd made a lot of conceptual progress and could definitely use some more horsepower 🐴🔋.

[nadavshatz]

@wycats happy to join this as well and maybe be able to help. I know that for our quite large ember app - poor engine support and poor micro-frontends support and modern tooling (think nx with webpack module federation) are probably the only reasons we're considering a full rewrite away from ember.

If we can help in any way avoid such a massive project i'd love to help make that happen.

[wycats]

@nadavshatz The convos I've been having with @richgt are basically in the vein of "Polaris Engines should be Ember's 'microfrontend' story." We've made some progress, both conceptually and practically.

If you're interested in joining a call (and anyone else reading this thread who has 🐴🔋 to offer), please ping me on Discord 😄.

[wagenet]

Just as an aside, I've been very aggressive with closing issues because I'm trying to get us a handle on things again and get stuff to a manageable state. The very fact that this issue already has multiple responses means we should reopen it again and that it's something worth pursuing further.

[wycats]

@wagenet I think you did the right thing in trying to get the issues under control. My philosophy is that the second someone raises their hand on an issue like this, it's a good enough reason to reopen (and to try to find a core team sponsor for the issue).

[phndiaye]

Really happy that this topic is getting some ideas and I would be glad to jump in if possible (I don't know exactly how Engines work internally though). I will ping you on Discord, @wycats, in case you setup a call soon :)

[wycats]

In case anyone's interested, @richgt and I have been exploring this topic at https://github.com/wycats/rsg-federation-spike.

It's purely exploratory, and we'll soon move on to looking at a proper proposal, but you can see some of what it will take to get this moving by looking at the commits 😄

[rsrini7]

As a library user, Our project also huge and looking for decouple ember engine from ember app. Any guidance ?

[villander]

cc @void-mAlex