jashkenas / backbone

Give your JS App some Backbone with Models, Views, Collections, and Events
http://backbonejs.org
MIT License
28.1k stars 5.39k forks source link

ES Modules: please discuss #4274

Open jgonggrijp opened 1 year ago

jgonggrijp commented 1 year ago

I suspect this is a feature many people would like, though perhaps not as many as were the case for Underscore. I personally think that there would be value in being able to import Backbone.Events, Backbone.Model, Backbone.Collection and perhaps the extend class emulator without importing the entirety of Backbone. Modularizing Backbone has been part of stage 2 of my maintenance takeover plan (https://github.com/jashkenas/backbone/issues/4244#issuecomment-993692119).

@vtalas started an attempt in #4215, but seems to have abandoned the branch since. As I will discuss below, I can think of a few good reasons why he might have given up.

I decided to make my own attempt, partly because I wanted to focus on regenerating a backwards-compatible UMD bundle using Rollup first, and partly because I didn't like Node.js's idiosyncratic .mjs extensions. I was planning to go a similar route as I have done for Underscore (see https://github.com/jashkenas/underscore/pull/2826, https://github.com/jashkenas/underscore/pull/2849 and https://github.com/jashkenas/underscore/pull/2914), with .js files in a modules directory that has a nested package.json with type: "module". However, I ran into serious difficulties. Unless some Rollup or ESM guru can help me out, I'm afraid we will have to face some hard choices.

Below is a link to my attempt-in-progress. The branch is not at all clean, so please do not base new work on top of it. Rollup would be invoked with npx rollup -c --bundleConfigAsCjs. https://github.com/jashkenas/backbone/compare/master...jgonggrijp:modules-2

The difficulties relate to the way the present, hand-written UMD bundle treats jQuery as an optional import.

I have tried and considered several solutions, but none of them is (1) treeshakable, (2) backwards-compatible and (3) sourcemap-tracable to the source modules at the same time:

So unless somebody can point me to a golden solution, I'm afraid we will have to make at least one of three sacrifices:

  1. Give up on treeshakability. This basically means giving up on ES modules entirely. While this is a valid option, it does not seem very future-facing.
  2. Give up on backwards compatibility. Also a valid option, but I feel that now is not the right time for a 2.0 version bump, so this would likely delay modularization substantially. This option would also mean figuring out an entirely new approach to the optional jQuery dependency.
  3. Give up on sourcemapping from the generated UMD bundle to the source modules. At first sight, this may seem like a minor sacrifice compared to the other two, especially given that Rollup keeps the bundle very readable. However, as I described above, every approach I can think of to make this sacrifice without the other two is rather iffy.

So, what do we do? Please share all the opinions, ideas and suggestions you may have!

chkpnt commented 1 year ago

Is UMD still a thing? IMHO ESM should get the primary focus nowadays.

jgonggrijp commented 1 year ago

Why do you think ESM should be a top priority?

UMD definitely is still a thing. Three main reasons:

Even if this were not the case, though, the main group to worry about is the existing Backbone user base. Breaking the interface so I can straightforwardly bundle the thing with Rollup will break the interface for everyone, not just the people who were still using UMD without an ESM emulation layer on top.

I welcome pull requests so we can discuss some concrete options; maybe someone will be able to think of a better solution than I managed so far.

chkpnt commented 1 year ago

Okay, my bad. I should not conclude from my use case to others.

As far as I remember, when I migrated the backbonejs application which I'm maintaining in 2020 from no module system with grunt concat to a module system with Webpack, ESM were already preferred over UMD. But maybe I remembered it wrong.

jgonggrijp commented 1 year ago

Yes, ESM is the standard and it has been for a while. Then again, not everyone is catching up at the same rate.

vilbergs commented 1 year ago

Would using the "exports" field in package.json help, more specifically the "conditional exports"? I haven't looked into the details, but on the surface it looks like it can point to different files based on the import mechanism used.

https://nodejs.org/api/packages.html#conditional-exports

I have also seen some packages simply keeping the ESM build separate in an "es" or "esm" subdirectory. So the import would look something like import { ... } from 'backbone/esm'. For better or worse it kind of implies a pipeline for the ESM build.

jgonggrijp commented 1 year ago

It will be part of the solution. However, for backwards compatibility, a UMD bundle would still be necessary.

kuwv commented 11 months ago

ESM and components are already widely used in many frameworks. I'd argue that right now I need Backbone to provide easy DOM management with the above from Vanillajs.

I don't need legacy support.

I'd say put the 1.x in a separate release LTS branch move forward. I understand that it would effect some compatibility with ecosystem but this is a good change.

jgonggrijp commented 11 months ago

@kuwv Could you clarify what you are proposing to do on the 2.0 branch?

kuwv commented 11 months ago

Sure

Then maybe other improvements such as graphql can happen.

jgonggrijp commented 11 months ago

@kuwv Please forgive me for quote-sniping; there is a lot to unpack here. I'll go through both of your posts:

ESM and components are already widely used in many frameworks.

Why mention other frameworks? Why could Backbone not do something that is different from other frameworks?

(I agree that there is a value to ESM and components, I just wonder why you feel that other frameworks are relevant here.)

I'd argue that right now I need Backbone to provide easy DOM management

I presume you mean that Backbone is currently doing this, too, but ...

with the above

... you want it to be ESM-based and integrate more with Web Components, ...

from Vanillajs.

... preferably without depending on jQuery?

What is currently lacking when you want to use Backbone with Web Components, and how does it bear relevance to ESM?

What problem do you think will be solved for you, if Backbone no longer depends on jQuery?

I don't need legacy support.

I'll take your word for that. Lots of other people do, though. Even when I change the interface, I want to keep migration feasible for most of those people.

* implement class support and deprecate 'extend'

ES classes are an even more difficult problem than ES modules. I agreed with Jeremy to postpone such a transition until a really good solution manifests itself. You can read #4245 for background. ESM will definitely happen sooner than ES classes.

* improve super initialization

What do you mean by this?

kuwv commented 11 months ago

It's fine.

I don't need legacy support.

I'll take your word for that. Lots of other people do, though. Even when I change the interface, I want to keep migration feasible for most of those people.

Let me clarify. An AMD/UMD based app does not need ESM support, as an ESM app does not need to support either AMD/UMD.

Backbone can be split into two supported versions: active and maintenance through the use of a "git workflows" with releases.

https://about.gitlab.com/topics/version-control/what-is-gitlab-flow/ or https://www.endoflineblog.com/oneflow-a-git-branching-model-and-workflow

Also, if SEMVER is done correctly an Alpha version can be used to prototype these ES6+ features without locking the project into it. This would allow implementation of classes and modules much more interactively with the community IMO. Supporting two versions also removes any pressure for legacy users to make immediate changes.

https://survivejs.com/maintenance/packaging/publishing/

That's why I think it wouldn't be beneficial to try to pack all package types and features within one build. It's just not needed. This is why versioning exists.

Why mention other frameworks? Why could Backbone not do something that is different from other frameworks?

When evaluating tools or frameworks I look at it's fits within an ecosystem. There are two primary use cases for backbone which are: vanillajs apps and microfrontend components. Backbone can be used with other frameworks to build smaller UI components that can then be assembled in something like React or Vuejs frontend.

https://single-spa.js.org/docs/ecosystem-backbone/

With microfrontends ESM/Comonents are actually viable approach with microservices internally (non-www). Ideally these systems want multiple UI components from each API it consumes to assemble pages.

Currently, backbone doesn't provide much help to developers here.

jgonggrijp commented 11 months ago

@kuwv I understand the concepts of semver and support branches. What is still unclear to me, is what changes you envision on the future-facing branch. Backbone as-is cannot be straightforwardly ported to ESM, otherwise I would have already done it.

I'm asking you a technical question, not a project maintenance question. How would you approach the optional jQuery dependency in an ESM-first restructuring of Backbone?

kuwv commented 11 months ago

@jgonggrijp Fair enough. I still believe it's important to be discussed here as this discussion is specifically about the builds.

As for your question, I'm sure SystemJS can be used from Rollup. The System.import supports conditional imports if the module exists in the path with transpilation. As a universal module loader it would not only provide CommonJS and ESM but can additionally allow AMD for jQuery. I feel like that's mostly there but might need some additional consideration for users.

Although, I would prefer if underscore would just provide the view capability backbone requires. But, I understand that would be a significant investment of time and effort.

jgonggrijp commented 11 months ago

Ah, I didn't think of SystemJS. I'll look into that, thanks for the tip!

kuwv commented 11 months ago

It seems ESM support is already coming to jQuery:

https://github.com/jquery/jquery/issues/4592 https://github.com/jquery/jquery/pull/5255

Also, a stopgap was provided:

https://esm.sh/jquery@3.7.1

jgonggrijp commented 11 months ago

That is certainly interesting. Just to be clear though, the problem with jQuery is not that it doesn't support ESM yet, but that Backbone currently has a "creative" approach to depending on it.

kuwv commented 11 months ago

Yeah, I've been looking at that too. The thread suggested https://www.npmjs.com/package/@rollup/plugin-node-resolve

But, that would be systems side.

VitorLuizC commented 10 months ago

If that's a block, maybe the solution is just an initialization function that receives $. So the consumers will provide it, instead of this auto-resolution


import * as Backbone from 'backbone';
import jQuery from 'jquery';

Backbone.setJQuery(jQuery);
jgonggrijp commented 10 months ago

Thanks for thinking along, @VitorLuizC. I think such a solution could work (and it would have a few other advantages), but it would be a breaking change. So that would amount to option 2, "give up on backwards compatibility".

tiagox commented 7 months ago

+1 to support ESM.

My main use case is consistency. We have a big project with a lot of dependencies only supporting CJS. It would be great being able to finally migrate all the CJS code to ESM to simplify bundle processes and to keep a consistente codebase (without having several ways to do each thing).

On a very personal note, the three caveats you've (@jgonggrijp) mentioned, are more likely a nice to have rather than must have.

jgonggrijp commented 7 months ago

@tiagox I always write all my code in ESM notation. The bundling tools add an emulation layer for the modules that are still written in CJS or AMD. What stops you from doing the same?

Treeshakability and sourcemap-traceability are non-optional, as far as I'm concerned. Without treeshaking there is no point in modularizing and without a sourcemap, you cannot see what's going on in your code. I can give up on backwards compatibility, and there might be no other option, but that means I have to wait and pool it with other breaking changes. I don't want major releases to be a frequent thing, users should be able to depend on some stability.

(Just to clarify, though: I'm still interested in magic solutions if anyone can think of one.)

valeriivolkovskyi commented 5 months ago

I would skip backward compatibility. There are no magic solutions; it is supposed to be changed to move forward. I would like to contribute with many things, but ES modules are a bottleneck. Personally, I love Backbone not because of stability, but also because of its simplicity and approach to building applications. In any case, in the future it will be necessary to remove jQuery as dependency

jgonggrijp commented 5 months ago

Thank you for your comment, @valeriivolkovskyi. There is a lot to unpack!

I would skip backward compatibility. There are no magic solutions;

I didn't mean magic magic, rather "something I have not thought of yet". That being said, if SystemJS is not the silver bullet and I don't find anything else (and I suspect that will be the case), then backwards compatibility is certainly out of the window.

it is supposed to be changed to move forward.

Could you elaborate a bit on this? What are you referring to with "it"?

I would like to contribute with many things,

Which things? If there are too many to list, could you just give a few examples?

but ES modules are a bottleneck.

Do you mean they prevent you from making those contributions? How?

Personally, I love Backbone not because of stability, but also because of its simplicity and approach to building applications.

❤️

In any case, in the future it will be necessary to remove jQuery as dependency

Why?

valeriivolkovskyi commented 5 months ago

I mean, I believe it's crucial to transition to ES modules as soon as possible. This transition will accelerate opportunities for contributions to the framework and enhance its modernity, provided that the community is interested in it.

Which things? If there are too many to list, could you just give a few examples?

I have some ideas to improve re-rendering performance. Or I would help with ES classes, or other issues. Do we have a roadmap BTW?

Why?

are you believe jQuery is still worth keeping as a dependency?

jgonggrijp commented 5 months ago

@valeriivolkovskyi Please excuse me for firing all these questions at you. I just want to get to the essence of what you are saying.

I mean, I believe it's crucial to transition to ES modules as soon as possible. This transition will accelerate opportunities for contributions to the framework and enhance its modernity, provided that the community is interested in it.

Do you mean "ES modules are the modern way to do things, so Backbone will look cooler if it uses them, which might attract new contributors"? Or is there more to it?

I have some ideas to improve re-rendering performance. Or I would help with ES classes, or other issues.

But only after Backbone switches to ES modules? For what reason(s)?

Do we have a roadmap BTW?

Somewhat. I previously published this list, which is mostly about Backbone's state of maintenance. We are currently at stage 3. I intended to do ESM in stage 2, but postponed it for the reasons discussed in this ticket.

Other than that (i.e., concrete features), there has been a vague road map in my mind. I will try to list it as concretely as possible over here:

  1. Don't change much unless it is necessary; I believe Backbone is mostly fine the way it is. Keep making small enhancements when they come up.
  2. Nevertheless, I anticipate some breaking changes that I was hoping to aggregate in a single major release:
    • Upgrade to Underscore 2.0, once I have released that and find it stable enough. (Underscore 2.0 will support Map, Set, iterators, promises, etcetera, and become stateless and properly treeshakable.)
    • Reconcile with the new TypeScript-inspired class sugar, if we ever find a good way. See the numbered list at the end of https://github.com/jashkenas/backbone/issues/4245#issuecomment-1002290690 for what I mean by "good".
    • Reorganize the way Backbone depends on jQuery and/or base Backbone.ajax on fetch by default.
    • Switch to ES modules (though I originally hoped to do this in a non-breaking way, as mentioned before).
  3. Now that jQuery 4 is in beta, this might inform future change in Backbone as well. This might go with step 2, or it might warrant another major release, if necessary.
  4. Resume just adding small enhancements.

are you believe jQuery is still worth keeping as a dependency?

I do, for the simple reason that a lot of Backbone's functionality depends on jQuery. Do you believe there is a need to get rid of jQuery as a dependency?

techcto commented 5 months ago

+1 remove jQuery -- Great talk -- I found myself refactoring my old backbone app, so glad this is going on. We just upgraded to the latest Webpack and swapped out all of the requires for imports. I am looking how to make the Router work with webpack ports, I am guessing an upgraded Router would also be nice.

jgonggrijp commented 5 months ago

@techcto What do you mean by an "upgraded" Router? What would you like to see changed about it?

kuwv commented 5 months ago

Do you mean "ES modules are the modern way to do things, so Backbone will look cooler if it uses them, which might attract new contributors"? Or is there more to it?

Let's not be dismissive. What backbone does well is provide a low barrier of entry to powerful SPA capabilities. Using standardized features does that better than esoteric or deprecated ones.

jgonggrijp commented 5 months ago

@kuwv let me get a few things out of the way.

jdittrich commented 3 months ago

If that's a block, maybe the solution is just an initialization function that receives $. So the consumers will provide it...

I know this breaks backwards compatibility, but this makes sense to me. I wonder if it is worth to explore if this would be easy to migrate to with a simple code change for backbone users or if it would create scenarios which get some stuck, unable to migrate.

jgonggrijp commented 3 months ago

@jdittrich For what it is worth, it makes sense to me, too. In fact, I think it is the most straightforward way to retrofit Backbone in an ESM jacket, if our aim is to keep Backbone otherwise the same as much as possible.

That is not necessarily my aim, though. Maybe I have been unclear about this. I want to offer the users stability that they can depend on, but if we're going to have a major update anyway, somewhere close to the library's 15th birthday, I'm thinking that maybe it should be as disruptive as possible. Pool as many breaking changes as we can think of, so that the next breaking update can wait another 15 years. It might as well be a full redesign? Thoughts welcome.

(And then continue to support Backbone 1.x for a while in parallel, as @kuwv has also suggested. And make sure there is a way in which Backbone 1.x and 2.x can be used together in a single application, to ease the transition.)

jdittrich commented 3 months ago

Pool as many breaking changes as we can think of, so that the next breaking update can wait another 15 years. It might as well be a full redesign? Thoughts welcome.

Gladly! I just wanted to stay on discussion here, and it seems most concerns were around maintaining backwards compatibility and transitioning to ESM. But maybe we want another issue for collecting potentially breaking changes? (or is there already one?)

jgonggrijp commented 3 months ago

@jdittrich Good suggestion, thanks. I just opened a new ticket for that purpose, see reference above. @kuwv and @valeriivolkovskyi, I encourage you to reply there, since you seemed to be full of ideas.

Yes, the concerns around here are about transitioning to ESM and backwards compatibility. I'm afraid those two cannot be reconciled, though I have not looked deeply enough into SystemJS yet to be sure. Then again, if you suggest a "less-breaking" conservative solution, I get the impression that this is what you prefer. Is it? There would be nothing wrong with that.

jdittrich commented 2 months ago

"less braking" conservative solution… get the impression that this is what you prefer.

Yes. At least when (for now) keeping Backbone mostly as-is, but transitioning to ESM, it seems that an "less-breaking", minor migration-effort-solution (like having to inject jQuery) might be a good solution.