JavaScript standard library contents

[littledan]

The JavaScript standard library is very minimal. Is this how it should be? What should we add?

Note, the purpose of this repository is to develop the infrastructure for built-in modules, not the contents. However, I know many of you have ideas for the contents as well, so let's brainstorm those in this issue exclusively.

[Mouvedia]

enhanced promises

• no leaking promises-aplus/promises-spec#179
• propagated cancelation
• splitting then into different methods

@littledan is this issue a dump for proposals? If that's the case I don't think we will ever be able to close it.

[ljharb]

Changing something that’s already in the language would need its own proposal; i think the standard library would be for adding new things.

[Mouvedia]

Changing something that’s already in the language would need its own proposal

@ljharb If the change is too radical or breaking, it has to be externalized: an alternative with another name to avoid confusion; future comes to mind in my case.

[littledan]

I meant this as a brainstorming thread. What we talk about here might graduate into proposals. As @ljharb says, let's try to focus on additions of new features, rather than changes to existing things (it's not clear how to expose what you're talking about as a built-in module). If this thread gets too unwieldy, we can close it.

[Mouvedia]

Composable?

[jasnell]

WHATWG URL should be normative. await JSON.parse() and await JSON.stringify() or similar

[NarHakobyan]

Would be nice to have promise cancelation

[Mouvedia]

uuid?

[Mouvedia]

Anything that validates ECMAScript internal stuff. e.g. https://mothereff.in/js-variables

[littledan]

@Mouvedia Would uuid make sense as part of WebCrypto? https://github.com/w3c/webcrypto/issues/215

[littledan]

@mathiasbynens 's Unicode RegExp properties should help with making that checker more efficient!

[Mouvedia]

That validator is just an example. My point is that the language should expose its own validators/assertors.

That would permit to have a basis for other interfaces (like the DOM). e.g. I had to make my own class/id assertors (html/css)

[Mouvedia]

@Mouvedia Would uuid make sense as part of WebCrypto? w3c/webcrypto#215

I don't know. Last time I checked the polyfill was enormous—it was almost impractical.

[littledan]

@Mouvedia Well, the idea would be that you just use uuid if you don't have the built-in one. (uuid actually uses WebCrypto in browsers today.)

[Mouvedia]

The idea of futures is something that should be pursued; in the same way that XMLHttpRequest was retroactively redefined in terms of fetch, we could have Promise be redefined in terms of Future where futures are more granular and hence give more control to the user.

@NarHakobyan https://github.com/tc39/proposal-javascript-standard-library/issues/16#issuecomment-447916074

[JonForest]

It would be great to be able to access object properties without having to do something along the lines of

const prop2 = obj && obj.prop1 && obj.prop1.prop2

Something like the lodash _.get function, or Angular's elvis operator in the templates (obj?.prop1?.prop2?) I'm not super-keen on _.get because it's turning the property path into a string, which is not great from a typing and build checking point of view, but I use it as an example of desired functionality.

[decadecity]

My two longstanding bugbears with the JS standard library (going back to my pre-jQuery days) are:

1. A better interface for dealing with cookies (i.e. doesn't involve directly manipulating a string - something like Web Storage).
2. A parser/constructor for URLs like Python's urllib.parse (other languages' standard library methods for dealing with URLs are also available).

For a language that spends a lot of its time running in a browser these seem like obvious candidates for a standard library.

[littledan]

@decadecity I'm wondering if the solutions at the web level, currently shipping in browsers, are enough for your use case.

1. Does Web Storage solve the issue for you, or are there remaining problems?
2. Does the WHATWG URL API solve this issue for you?

[littledan]

@JonForest Have you see the optional chaining proposal? It's attempting to add that operator.

[decadecity]

@littledan

1. Web Storage is fine for client side storage but doesn't do cookies :stuck_out_tongue: Directly manipulating the document.cookies string is a terrible API, I'd like to see something like the extension cookies API.
2. The URL API looks like it's about right, thanks :+1:

[littledan]

@decadecity Work is ongoing to evolve the cookie API itself at https://github.com/WICG/cookie-store . TC39 is probably the wrong place to advance work in this area, as we focus on things that don't involve I/O. Cookies don't mean much outside of the Web.

[anotheredward]

In terms of what we can see people using, lodash is the most depended upon library that would be potentially appropriate to exist in core: https://www.npmjs.com/browse/depended

It's a bit harder to break down which functions are used the most, but you can get an idea here: https://www.npmjs.com/search?q=lodash&ranking=popularity

There is also a minimal version of the library: https://github.com/lodash/lodash/wiki/Build-Differences

[StoneCypher]

No, the JS standard library should not be minimal. Part of the reason there's so many variations on the underscore theme is that there's a lot of really basic stuff missing from the language (like deep copy.)

Unfortunately it's not clear where feature requests belong. While I was writing out a flood, I was told "not here. Maybe on issue 16."

There's dozens of easy one-liners that would massively improve the language if gatekeepers would permit them

[StoneCypher]

I'll copy pasta a bunch right now if someone tells me where to put them

[littledan]

@StoneCypher I'm glad you came here to do additional brainstorming for contents. For JavaScript one-liners, maybe could you make your own repository or gist for development and link to it from here?

[martinheidegger]

Somehow I feel like the input from @kgryte could be useful to this discussion ;)

[StoneCypher]

Would be really nice if there was an official place to give people recommendations for a more complete standard libary

[StoneCypher]

There's a whole lot of donation languishing in repos that gatekeepers don't look for

JS could be an amazingly easier language with very little work on anyone's part if the gatekeepers would facilitate donation

[StoneCypher]

In erlang, for example, there is a bug tracker where I can go, add a ticket with some code and some appropriate tags, and if my code isn't garbage and makes sense, there's a good chance it'll show up in the next standard library under some very different name

Recommendations I gave to Ian Hicks almost 20 years ago and gave code for are being invented this year and treated as important and ground breaking ☹️

Javascript should have a process for letting people who aren't friends with standards maintainers be helpful too

[StoneCypher]

I'm just going to go ahead and put what I wrote yesterday into a ticket

If I put it in my own repo nobody will ever look at it and it will die on-site

[jhpratt]

I'm surprised no one has brought this up yet — the much-proposed range generator function would be a great addition, and presumably without much opposition. Following Python's semantics would make sense.

[ljharb]

@jhpratt Array.build was proposed long ago, and might be a candidate for picking up again.

[jhpratt]

@ljharb I think it would be better to have it as a generator to avoid memory constraints, no? Though having a map function would be useful in certain cases, I imagine.

[lachlanhunt]

More useful functions for working with generators and iterables should be included.

Standard sequence generators:

• counter(start = 0, step = 1) - Simply generates values starting from the given start value, incrementing by step. This one is most useful when combined with

• range(start, end, step) - generates a sequence from start to but not including end, incrementing by the value given in step (default 1 or -1 depending on whether start < end)

  for (let i of range(0, 10)) { ... }

• random() - yields pseudo random values. Though this one seems to be covered by an existing proposal https://github.com/tc39/proposal-seeded-random

Generator modifiers:

Some of these are based on existing Array or lodash/underscore methods for Arrays, which are also useful for generators so that they can be run lazily as items are pulled from the generator, rather than first converting to an array.

• map(it, mapFn)
• forEach(it, fn)

• take(it, n) - Wraps a generator to limit the number of items that can be taken from it. This is particularly useful with generators that produce infinite sequences.

  [...take(map(random(), mapToInteger(0, 10)), 10)] // Creates array of 10 random values between 0 and 10. (mapToInteger function not illustrated, but assume it converts a random float to an integer)

I've implemented these, and more, in my own library. I've listed just some of the most useful for inclusion in a standard library.

https://github.com/lachlanhunt/generator-utilities/ (Not everything in there would be appropriate for inclusion, though)

Edit: Fixed syntax error in code sample.

[viktor-ku]

If you have general range it isn't clear what type it's gonna produce: array, maybe set, maybe weak set or any other current or future iterable? Why is it worse in your opinion than Array.range() or Set.range() hence adding methods to Array or Set directly without having to do this in standard lib? The same goes with len or take or map.

Question then arrives isn't it better to rename this proposal to enhance js builtin-ins?

[StoneCypher]

It's really unfortunate that Dan keeps closing my tickets without taking the time to understand them first, even though the ones that got talked over had community support :(

I am not alone in feeling that gatekeeping in JS is out of control

[ProdigySim]

1. Get data from NPM and github looking at package.json dependencies and subdependencies
2. Find the most depended-on libraries
3. Filter by human to determine which ones would fit in a standard library

I think a Standard library for JS should seek to alleviate npm dependency issues.

Having a standard implementation for basic functionality that works in both Node and Browser environments would remove much of the need for simple NPM packages that increase the number of potential security events just by nature of distributed ownership.

[lazarljubenovic]

A bunch of things missing from Array's prototype:

• flat, flatMap (although it's a stage 3 proposal now)
• chunk(count: number): T[][], makes [1, 2, 3, 4, 5] into [[1, 2], [3, 4], [5]] with argument 2
• remove(index: number)
• takeUntil(predicate: (t: T => boolean))
• takeWhile(predicate)
• dropUntil(predicate)
• dropWhile(predicate)
• the right version of the above four, which are used for pruning from either side of the array until or while a condition holds for sequential elements
• partition, like a filter which returns two disjoint arrays
• unique, uniqueBy to remove duplicates
• filterIndex would be to filter what findIndex is to find\
• scan, a reduce which reports partial solutions on each iteration

Set operations are missing too (edit: been notified of an existing proposal). Since these are n-ary (n > 1), instead of methods like a.op(x).op(y) or a.op(...xs), static functions might make more sense:

• Set.union(...sets)
• Set.union(...sets)
• Set.difference(set, ...sets)

Not sure if Array deserves these too.

---

Sorting with the default coercion to string is ridiculous; [111, 22, 3] is already sorted by default because strings are lexicographically ascending. Maybe some global common comparison functions? Eg. for numbers, [111, 22, 3].sort(Compare.number) where Compare.number is (a, b) => a > b ? -1 : 1).

Probably a sortBy would also come in handy so we don't have to arr.sort((a, b) => a.x.y.z > b.x.y.z ? -1 : 1) and instead do arr.sortBy(a => a.x.y.z).

---

Add more data structures: Queue, Stack would be a good starting point, where implementations could choose more efficient storage strategies (eg. linked lists) instead of working with arrays and push/pop/unshift/shift.

Of course, LinkedList, Tree and Graph would be great addition as well.

The constructor of linked lists could tweak the exact type, with switches like circular and/or doubly-linked (or just introduce more constructors). Then we get things like get head, get tail, add to head, add to tail, remove from head, remove from tail, add after, add before.

Trees are interesting because the traversal of the DOM has improved a lot and it's quite pleasant to work with with all the things you can do to nodes. A lot of these operations have little to do with DOM-specific API and a lot to do with trees in general (parent, children, insertBefore, insertAfter, remove, contains, etc). They would be a starting point for more things to add to standard library such as BinarySearchTree, BTree, BPlusTree, etc, which would have all the operations you expect them to have.

Similar with graphs, lots of common operations to add, including things like loading and serializing to a matrix, etc.

---

By the way, I think we shouldn't talk about a single "standard library", but instead several of those. I mean, aren't Map, Set, Math, URL, etc. already "standard libraries"? They provide utilities for common tasks. Even Array, if it wasn't so tied to the low-level working of [] would be just a utility for working with arrays -- you could still write a for loop to Array#reverse.

[ljharb]

For set operations, see https://github.com/tc39/proposal-set-methods

[joelgallant]

Is the intention to bring any existing standard library into this import style?

I like this idea, but it would feel inconsistent if a subset of lodash, say, were to be added to the language in a different (import reserved path) way than others, like Math.

[lazarljubenovic]

I like this idea, but it would feel inconsistent if a subset of lodash, say, were to be added to the language in a different (import reserved path) way than others, like Math.

We could add the import style for Math as well and alias it back to global's Math?

[emilianobovetti]

There are so many implementations of Maybe/Option and Either/Result monads that I feel a standard library could be perfect fit for those.

[renatoagds]

@littledan Immutable API? I think we could take a look in Immutable.js and bring some ideas from there. We'll have a crash between Map and Set that's already implemented in JS, but could be a good point of start.

[jbreckmckye]

Like @emilianobovetti, I'd like better ways to handle non-existent values, perhaps using a Maybe monad.

Consider an optional object with recursively optional fields:

// Could be one of three things:
// 1. undefined
// 2. {name: 'Joanna', age: undefined}
// 3. {name: 'Joanna', age: {years: undefined}}
const user = await getUser();

const yearsOld = user.age.years; // will break 

If user was wrapped in a Maybe, we could query and manipulate its fields with no fear of typeErrors:

const maybeUser = await getUser();
const yearsOld = maybeUser
    .map(_ => _.age)
    .map(_ => _.years)
    .getOrElse('Not known');

This may become less compelling if tc39/proposal-optional-chaining advances from Stage 1.

[bmarkovic]

@jbreckmckye I don't see why optional chaining syntax couldn't be employed on "whatever-optional-type-is-chosen" to behave like similar operator does in Rust, and to behave as proposed on non-monadic objects (ie. return undefined). It would be semantically consistent, even if perhaps not fully transparent to new users at first, but then, it can't be less transparent to new users than optional types and Promises.

Speaking of Promises I personally also liked the idea of the ? operator returning undefined and discarding rejection when applied to awaited Promises i.e.

// if Promise is rejected result = undefined, move along
let result = await someFnThatReturnsPromise()? 

[tristan-shelton]

I would love to see deep get/set/copy as part of the standard library.

[lazarljubenovic]

The problem with deep copy is that it's difficult to determine how deep and what exactly you want to do. How do you "deep copy" a function? Do you carry the prototype into the copy? What about cloning things like DOM nodes? Not to mention circular structures -- graphs? trees? doubly linked lists? The only "proper" way to deep copy is having something along the lines of a copy constructor for everything (including authored classes).

[maple3142]

How about having a function like assignDeep? (It will work like Object.assign, but deeper one.)

[ljharb]

I think a “structured clone” protocol might pave the way for “deep copy” operations without having to bake in answers for those questions.

[tristan-shelton]

@lazarljubenovic While I do agree those issues should be thought through I think that any model you pick would be less confusing for most developers than the shallow copy that we get as part of the standard library today.

There are also many reference implementations of deep copy/clone out there in the wild being used by legions of developers so those would probably be a good place to look for answers to some of those questions.