search

jashkenas / underscore

JavaScript's utility _ belt
https://underscorejs.org
MIT License
27.3k stars 5.53k forks source link

The Big Kahuna: Underscore + Lodash Merge Thread #2182

Closed jashkenas closed 9 years ago

jashkenas commented 9 years ago

Dear Everyone,

JDD got in touch with me over email this morning, inquiring about the possibility of merging Underscore and Lodash together. I think it's high time to talk about the possibility again.

Some initial points for discussion:

Edits in response to comments below:

Although it would be nice to keep the bulk of the discussion here ... there's also a Contributors-only cross-thread here: https://github.com/jashkenas/underscore/issues/2184

jdalton commented 9 years ago

Before digging into the discussion I just wanted to say :+1: and thank you to Jeremy for hosting and being open to discussions.

Sidenote: For those interested there's also a fair amount of chatter on twitter on the merge.

tyscorp commented 9 years ago

Does that mean lodash will no longer follow semver?

jamesplease commented 9 years ago

For future commenters: many of you will :+1: to this. If that's all you've got to say, it might be best to not add noise to the conversation. There's quite a bit to be discussed before this could ever happen, and it seems like this issue is attempting to be the forum for that discussion. If you want to stay involved, click the "Subscribe" button to the right to see notifications.

phpnode commented 9 years ago

Does that mean lodash will no longer follow semver?

@tyscorp I'm going to go ahead and say no, that won't happen

lennym commented 9 years ago

Firstly :+1:

Now in actual response to a few of the points raised (disclaimer - I am a lot more familiar with underscore than lodash):

Call it underscore or something completely new. Lodash is obviously a take on underscore, and doesn't make a lot of sense on its own.

Things I would like to see from lodash in underscore:

Code split by category/method like var chunk = require('lodash/array/chunk'); in the lodash readme. I think this could go some way to mitigating the fact that lodash is much larger code-wise.

_.get - I find this absurdly useful, and it's basically the reason I started installing lodash instead of underscore.

Probably more to come as I think of things...

Ky6uk commented 9 years ago

Lodash is currently > 12k lines of code, and Underscore is 1.5k.

How about comparing only existing functions in both of __? Ofc we can make a custom build of lodash if 12k is our problem.

JasonGhent commented 9 years ago

"Lodash is currently > 12k lines of code, and Underscore is 1.5k. I understand that part of that is JSDoc noise, but it still seems potentially problematic and worth talking about."

Can we categorize what else contributes to the size differential?

tdd commented 9 years ago

Hey all,

I obviously :+1: this to death. Thanks @jashkenas for re-opening this, let's hope it's not a can of worms anymore.

A couple notes off the top of my head.

One thing from Lo-Dash I'd love to see happen in Underscore is the code split (that you find as multiple modules in the amp.js ecosystem) and general build/tooling that lets us not bundle the whole code when browerify'ing / webpacking / brunching / etc. a small lib of ours that only uses a couple _.xxx methods.

I wasn't aware of LoDash's _.get, which is admittedly kinda sweet.

Underscore would also need to be way more open about the iteratee thing, and use both situations (callback vs. prop name) in its code/usage examples, to drive the point home. It's great that it took this page from LoDash's book, but perusing the docs, it's almost completely hidden. It only shows up once and then is never illustrated again. This makes for way shorter code, though, and ties neatly with _.result, too.

Let's move this forward!

zacechola commented 9 years ago

The Name: "Lodash" is cute, but keeping "Underscore" makes the most sense, both for historical and _ variable reasons.

I hear iojs may be back on the market. :stuck_out_tongue_winking_eye: But seriously, either name is fine by me.

Lodash is currently > 12k lines of code, and Underscore is 1.5k. I understand that part of that is JSDoc noise, but it still seems potentially problematic and worth talking about.

As has been pointed out already in this thread by @Ky6uk, there needs to be an apples to apples function comparison here.

Documentation style. Ideally I'd like the Underscore docs to be written even more conversationally than they are at the moment.

:+1:

We should keep nicely annotated source code as a priority.

:+1:

And then the actual main work — I'm not familiar with the Lodash codebase. What parts of it do folks have objections to merging? What might not fit in well within a merged Underscore umbrella? Anything? Maybe not?

Versioning needs to be clarified for sure and I also would personally appreciate the modularization that @tdd brought up as a nice to have.

wraithgar commented 9 years ago

The only pain point we've noticed using lodash is that in some packages that have several modules, each with quite a few lodash dependencies under them and in their dependencies, npm dedupe has a lot of trouble and often silently exits while trying to run. This is of course not currently a problem with things using underscore because it's one single module, and is something worth at least mentioning if you're talking about a transition to the lodash individual module approach.

aredridel commented 9 years ago

npm 3 will help the dedupe process immensely.

glesperance commented 9 years ago

Would be great if the merge could keep using semantic versioning as in lodash

kud commented 9 years ago

To be honest, I was already afraid with nodejs / iojs dilemma but glad they'll merge too. So I couldn't be agree more to see your projects merge too.

Congratulations anyway to you, you make a really good work.

(warning: I mostly use lodash for the moment on all my projects)

+1 for "underscore" because history + this is what I say when I read it.

+1 for the underscore's doc, clearer/cleaner in my opinion.

xcoderzach commented 9 years ago

I'm a huge fan of how lodash does lazy evaluation chains by default, rather than explicitly needing to call .chain(). Ideally that would be the standard going forward.

mathiasbynens commented 9 years ago

:+1: Let’s make lodash ∶ Underscore ∷ io.js ∶ Node.js and merge already!

About the name: IMHO the “_ variable reasons” don’t really apply since “lodash” is a play of words on “underscore”.

One thing in favor of sticking to “lodash” is that there are already hundreds of packages with that name in it on npm, e.g. https://www.npmjs.com/package/lodash.foreach. Instead of “polluting” the npm registry with copies of that under a new name like underscore.foreach (i.e. deprecating hundreds of packages and creating new copies instead, only for the new name), it might make more sense to just deprecate the one “underscore” package in favor of “lodash” and the related packages.

bjmiller commented 9 years ago

Both names are highly depended on. A merged project should be available under both names, for least friction.

rafde commented 9 years ago

So would lodash function behaviors be a super set of underscore? Or the other way around? I tried to replace underscore with lodash 3.x and a function like _.extend differ. Underscore will copy all props (even prototypes) while lodash only copies hasOwnPropertys. But I suppose that's what underscore 2.x will be, the breaking change.

alexindigo commented 9 years ago

What the governance model would be?

alexindigo commented 9 years ago

Re: @mathiasbynens

And as for the naming, how about keeping underscore for the lightweight version of lodash, like basic / core module, but the whole ecosystem will be lodash thing.

mathiasbynens commented 9 years ago

@bjmiller

Both names are highly depended on.

Agreed.

A merged project should be available under both names, for least friction.

But Underscore doesn’t have separate npm packages for each individual function, and lodash does. So, there would be no friction at all by just continuing to refer to the existing lodash modules. Creating copies of those on the other hand, would.

spadgos commented 9 years ago

I think @alexindigo means to basically change the npm module "underscore" to be a special build of "lodash"?

neberej commented 9 years ago

+1

etc. Oh this is exciting.

buddy-sandidge commented 9 years ago

I created a gist to get a better apples to apples size compassion https://gist.github.com/xbudex/a03aaf17ab8f984c18f0

To try to get a more apples to apples compairison, this is what I did.

The results:

Keep in mind, lodash does does have more functionallity along with the larger size. There are probably flaws with this method, but it gets us closer to an apples to apples compairison.

calvinmetcalf commented 9 years ago

based on browserifing a file with module.exports = require('lodash|underscore');

regular minified minified + gzipped
underscore 53515 16690 5895
lodash 406335 51729 18942

commands were

browserify script.js | wc -c
browserify script.js | uglifyjs -mc | wc -c
browserify script.js | uglifyjs -mc | gzip | wc -c

edit correct size for lodash

samiconductor commented 9 years ago

Thanks for creating the most used modules!

basicallydan commented 9 years ago

The Name: "Lodash" is very cute, but keeping "Underscore" makes the most sense, both for historical and _ variable reasons. Because npm doesn't allow name redirects (AFAIK), the merged project would be available as a module under both names for the foreseeable future.

Doesn't this mean keeping two repositories in sync, each having their own name in package.json? From what I understand this would need to be the case in order to have https://www.npmjs.com/package/lodash and https://www.npmjs.com/package/underscore continue to work. Am I wrong?

This is great, by the way!

rtorr commented 9 years ago

Could underscore (on npm) just become an official build of lodash (like lodash-underscore build used to be)? Merge docs as well.

wheaties commented 9 years ago

Have to agree on the comment "lazy by default" of Lodash. That right there helps immensely with many normal use cases. And _.get.

benlesh commented 9 years ago

There's no need to fear, UnderDash is here!

seriously, name it "UnderDash".

fibo commented 9 years ago

It does not make sense imho, unless you find a very smart way to import lodash code into underscore. I am sure a great coder like you can find a clever solution, if yes it does not make sense to rename packages, underscore would be a monolithic distribution of lodash functions, lodash will be a modular system that extends underscore.

nemo commented 9 years ago

It would be interesting to see if you can modularize the libraries and help Javascript SDK builders.

In the real world, you only ever use a handful of lodash/underscore functions and it would be lovely to abstract them out so they can be required explicitly. Much like the way component has it split up; e.g. each.

But that's probably too big of an architectural change to be discussed right now.

sch commented 9 years ago

@jashkenas — I'd love for you to talk a bit more about how you'd like to make the docs more conversational.

I think Underscore was essential in promoting functional programming in the JS community, and a lot of that can be owed to great examples and conversational descriptions. Do you envision there being more of a tutorial component to the docs? Maybe keep a single-page reference style document for the main page, but have more detailed pages of history, explanations and examples (immutable.js-docs-style)? Do you plan on bringing in more of the underscore-contrib tool belt?

jashkenas commented 9 years ago

I'm sure y'all are busy, and it will take some time to respond ... but I'd love to hear thoughts about this and the above issues from the core contributors over the years. So, pinging: @braddunbar, @megawac, @akre54, @jridgewell and @michaelficarra.

donnyv commented 9 years ago

I don't think they need to merge. I like Lodash's streamlined api and there api documentation is better. No thanks!

mLuby commented 9 years ago

+1 underdash

Would also love some laziness.

jashkenas commented 9 years ago

I think Underscore was essential in promoting functional programming in the JS community, and a lot of that can be owed to great examples and conversational descriptions. Do you envision there being more of a tutorial component to the docs? Maybe keep a single-page reference style document for the main page

I really like open-source libraries with a single, internally-linked, HTML page for documentation. Command-F is the best search tool you have for docs. It's instant, and you're always on the right page to find what you need.

Perhaps some JavaScript fanciness to collapse the sections that you don't want to look at.

A general tutorial section would be a great addition. "More conversational" mainly means just writing out the docs with a little more prose, humor, and friendliness. Ideally, if you're unfamiliar with the library, you can read through them in sequence if you want to, and not feel like you're reading a dry computer-generated manual.

sch commented 9 years ago

Ideally, if you're unfamiliar with the library, you can read through them in sequence if you want to, and not feel like you're reading a dry computer-generated manual.

Absolutely. Do you suppose this is a good sidequest to go on separate from this merge? Worth opening a separate issue for?

idibidiart commented 9 years ago

+1 from the Technology Minimization Committee ... too many confusingly similar JS libraries and this would set a nice precedent for teams from competing libraries working together ! yay! like Io.js and NodeJS =) It's officially a trend now.

austinpray commented 9 years ago

Underscore should just become the underscore build of lodash. Lodash is way better maintained than Underscore.

  • The Name: "Lodash" is very cute, but keeping "Underscore"...
  • Lodash is currently > 12k lines of code, and Underscore is 1.5k...
  • Documentation style. Ideally I'd like the Underscore docs...
  • We should keep nicely annotated source code as a priority...

This is all really low priority stuff compared to the objective benefits of lodash

akre54 commented 9 years ago

Eh. I know this won't be a popular opinion, but in my experience Underscore's small and readable code is a huge boon in my projects and in my understanding of the language of coding in javascript, rather than details of its implementation engines.

I also don't particularly like the way JDD runs Lo-Dash. I think it'd be a shame to lose Underscore to the enterprisey style of the Lo-Dash project.

In a perfect world, I'd rather see Underscore adopt more regular releases, a better versioning system (maybe not semver, but something), and more stringent testing. All things must grow up, but this is kind of sad to me.

ghost commented 9 years ago

to prevent all the +1s, you can just lock the issue so only collaborators have access to add comments (though that might mean adding the lodash committers who should be involved to the collaborators repo).

Locking the conversation means:

  • Other users can't add new comments to this issue.
  • You and other collaborators with access to this repository can still leave comments that others can see.

You can always unlock this issue again in the future.

idibidiart commented 9 years ago

@austinpray what is happening here is far bigger than which is better.... you have two competing and very popular JS libraries talking about making things simpler for everyone. Obviously, I expect the improvements from lodash to make it into the merged library

ConAntonakos commented 9 years ago

Awesome! This is fantastic news.

Having to juggle and decide whether one should use underscore or lodash as their functional tool belt can now finally be quashed. Thank you.

Will this be a complete merge or will certain parts be excised for example?

schmod commented 9 years ago

I support this merge.

Given that some of us are still suck managing JavaScript dependencies via <script> tags, I'd like to see "Underscore" maintained as a well-defined set of core dependencies, which would be a subset of the full Underscore/Lodash library.

This way, 3rd-party libraries that still need to support users without a modern dependency-management system can have a convenient shorthand for declaring that they require "Underscore," without forcing users into the full lodash build, or having to enumerate every single lodash micro-module that the library depends on...

I also don't mind if this core library receives few updates. Part of Underscore's beauty is its stability, simplicity, and small size.

jashkenas commented 9 years ago

Eh. I know this won't be a popular opinion, but in my experience Underscore's small and readable code is a huge boon in my projects and in my understanding of the language of coding in javascript, rather than details of its implementation engines.

I also don't particularly like the way JDD runs Lo-Dash. I think it'd be a shame to lose Underscore to the enterprisey style of the Lo-Dash project.

In a perfect world, I'd rather see Underscore adopt more regular releases, a better versioning system (maybe not semver, but something), and more stringent testing. All things must grow up, but this is kind of sad to me.

It's a good thing that this isn't a democracy, nor is it a popularity contest ;) It's the wild west of GitHub. That's a very valuable opinion, so thanks for sharing.

I think the idea is to talk about the possibility of having a small(ish) and readable code base, and a non-enterprisey development style, while still potentially going ahead with a merge. That doesn't seem like an impossible thing to achieve.

tracker1 commented 9 years ago

I would suggest making lodash have a single reference of a peerDependency that equates to underscore >=currentUnderscoreVersion and that it's index in npm basically become module.exports = require('underscore') which is as close to npm redirecting to another module as it gets... this way lodash doesn't need to be updated in lockstep, and when npm3's node_modules flattened structure comes out, it'll avoid duplication of code.

akre54 commented 9 years ago

I think the idea is to talk about the possibility of having a small(ish) and readable code base, and a non-enterprisey development style, while still potentially going ahead with a merge. That doesn't seem like an impossible thing to achieve.

Definitely. But historically these haven't been JDD's goals. And he has very strong opinions on these kinds of things. I guess I'm saying I'll believe it when I see it.

shama commented 9 years ago

but I have no problem praying to this golden cow to keep the philistines happy ;)

That doesn't exactly instill confidence that code won't be published without a more vigorous vetting and things like https://github.com/jashkenas/underscore/issues/2061 and 1.8 won't happen again.

Will @jdalton have control of the releases? I primarily use lodash because I trust @jdalton more, no offense intended to @jashkenas.

schmod commented 9 years ago

I personally prefer Underscore's docs. The documentation is nicely-formatted, and very concise. The annotated sources are also fantastic.

I think that there is almost definitely an achievable middle-ground between Underscore and Lodash's documentation style.

I'm not familiar with lodash's development process, so I don't know what makes it "enterprisey." If "enterprisey" means semver, code reviews, and test coverage, I'm all for those things.

ilovett commented 9 years ago 
underscore-lite = underscore
underscore = lodash

boom, done