Example of the utility of static private methods

[domenic]

This is loosely based on existing code in jsdom, with some additions to illustrate further points.

I am working on a class, call it JSDOM, which has some static factory methods:

export class JSDOM {
  // ... elided ...

  static fromURL(url, options) {
    if (options.referrer === undefined) {
      throw new TypeError();
    }
    if (options.url === undefined) {
      throw new TypeError();
    }
    options.url = (new URL(options.url)).href; // normalize

    // ... TODO: actual interesting stuff
  }

  static fromFile(filename, options) {
    if (options.url === undefined) {
      throw new TypeError();
    }
    options.url = (new URL(options.url)).href; // normalize

    // ... TODO: actual interesting stuff
  }
}

I notice two things:

• The options preprocessing is distracting from the main important part of the method.
• Some of my options preprocesing is shared between both classes.

Both of these are things that are addressed well by the extract method refactoring.

I have two choices on how to do this:

1. Extract into actual static methods of the class.
2. Extract into functions at module-level.

Although it is a matter of opinion, I find (1) better than (2), because it keeps the code inside the class that it's designed for, and near the methods that use it, instead of somewhere later in the file. Today, (1) has the disadvantage of polluting my public API, so I usually choose (2). But if we had static private methods, I could do (1) without that drawback.

Still, let's assume that arguments from #1 prevail and we don't get static private methods. It's not too bad:

export class JSDOM {
  // ... elided ...

  static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);

    // ... TODO: actual interesting stuff
  }

  static fromFile(filename, options = {}) {
    normalizeOptions(options);

    // ... TODO: actual interesting stuff
  }
}

function normalizeFromURLOptions(options) {
  if (options.referrer === undefined) {
    throw new TypeError();
  }
}

function normalizeOptions(options) {
  if (options.url === undefined) {
    throw new TypeError();
  }
  options.url = (new URL(options.url)).href;
}

But as I continue my work on implementing these functions, the lack of static private ends up felt more deeply. I get to the following point:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;

  #registerWithRegistry() {
    // ... elided ...
  }

  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);

    const body = await getBodyFromURL(url);
    const jsdom = new JSDOM(body, options);
    jsdom.#createdBy = "fromURL";
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }

  static fromFile(filename, options = {}) {
    normalizeOptions(options);

    const body = await getBodyFromFilename(filename);
    const jsdom = new JSDOM(body, options);
    jsdom.#createdBy = "fromFile";
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}

Again I notice I have some duplicated code. Here is the code I want to write to clean this up:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;

  #registerWithRegistry() {
    // ... elided ...
  }

  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);

    const body = await getBodyFromURL(url);
    return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromURL");
  }

  static fromFile(filename, options = {}) {
    normalizeOptions(options);

    const body = await getBodyFromFilename(filename);
    return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromFile");
  }

  static #finalizeFactoryCreated(jsdom, factoryName) {
    jsdom.#createdBy = factoryName;
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}

But I can't, because static private methods don't exist. Here is the code I have to write to work around that:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;

  #registerWithRegistry() {
    // ... elided ...
  }

  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);

    const body = await getBodyFromURL(url);
    return finalizeFactoryCreated(
      new JSDOM(body, options), "fromURL",
      (jsdom, value) => jsdom.#createdBy = value,
      (jsdom, ...args) => jsdom.#registerWithRegistry(...args)
    );
  }

  static fromFile(filename, options = {}) {
    normalizeOptions(options);

    const body = await getBodyFromFilename(filename);
    return finalizeFactoryCreated(
      new JSDOM(body, options), "fromFile",
      (jsdom, value) => jsdom.#createdBy = value,
      (jsdom, ...args) => jsdom.#registerWithRegistry(...args)
    );
  }
}

function finalizeFactoryCreated(jsdom, factoryName, createdBySetter, registerCaller) {
  createdBySetter(jsdom, factoryName);
  registerCaller(jsdom, registry);
  return jsdom;
}

This code is basically worse than the original code with duplication. The lack of static private, and the workarounds I have been forced to employ to get around it, have made persisting with duplication a better choice than attempting to apply extract method---because extract method has basically been broken.

---

This is why I think static private methods are quite important. Without it you cannot apply basic refactoring patterns, which users will expect to be feasible, to JS classes.

[littledan]

@zkat @ljharb Do you find these scenarios relevant or useful?

[ljharb]

@littledan not to me personally - i'd use "Extract into functions at module-level" every time - but I think the OP is a well-motivated justification for static private (the first I've seen).

[domenic]

"Extract into functions at module-level"

As shown in the second half, that doesn't really work if you need access to private state.

[ljharb]

Very true; i was trying to come up with an example I’d prefer, but the access to instance private state is the kicker.

[littledan]

Seems like a key aspect here, which I missed in explaining the feature in the past, is that it's especially likely to come up with factory methods. I think there's a not quite exactly so bad way to split it up, by using instance private methods for the "second half":

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;

  #registerWithRegistry(registry) {
    // ... elided ...
  }

  #init(factoryName) {
    this.#createdBy = factoryName;
    this.#registerWithRegistry(registry);
    return this;
  }

  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);

    const body = await getBodyFromURL(url);
    return new JSDOM(body, options).#init("fromURL");
  }

  static fromFile(filename, options = {}) {
    normalizeOptions(options);

    const body = await getBodyFromFilename(filename);
    return new JSDOM(body, options).#init("fromFile");
  }
}

Yes, this is splitting in two, and if you're refactoring between public and private static factory methods, it would be not so much fun. Maybe there are other cases which split up even worse, and maybe it's unintuitive how to do the refactoring I did above, but any case I can think of divides up into two pieces more or less--before you create the instance (function outside the class) and after (instance private method). The final code, in this particular case, doesn't look so bad to me.

Does anyone have any use cases for private static fields?

[domenic]

Does anyone have any use cases for private static fields?

Only personal-preference ones, analogous to the (1) vs. (2) from the first half of my post.

[zkat]

I agree that this is a compelling reason to support them! Thanks, domenic :)

[littledan]

@allenwb @erights What do you think about this motivation for static private methods?

[erights]

I favor the variant of (2) that @allenwb suggested: support lexical declarations at the class level. This is like extracting to a function at module level, except that it is in the scope of the private instance field names, and can therefore access that private state on its arguments.

What ever happened to that suggestion? Did anyone ever turn it into a proposal? Did it ever get disqualified for some reason? Why?

[domenic]

I would be happy with such a variant. Especially if the way you declared such lexical declarations was by using static #foo() { } and static #bar = baz; ;).

[ljharb]

@erights to clarify: static publics, and instance publics and privates, would continue as proposed, but static privates would be covered by lexical declarations (presumably with var/let/const) at the class level?

[ljharb]

Wasn't the declaration static what creates the inheritance confusion in the first place that demoted statics down to stage 2?

[littledan]

@erights My concern (@wycats convinced me of this) with @allenwb 's proposal: Keywords like function and let don't exactly scream, "unlike the other things in this list, this declaration is not exported and it's also not specific to the instance". I think something that is more the form of a field or method declaration would be a better fit for a class body and more intuitive.

I like what @domenic is suggesting: using me of syntax for a lyrically scoped function or variable. I presented something like this initially as the semantics for both instance and static private methods (not my idea; it was derived from earlier comments from @allenwb and others. See this patch which switched to doing type checks). Several committee members said in issues (#1 #3) that would prefer type checks on the receiver rather than the lexically scoped semantics.

If we did that sort of change, there are some semantic details that I'm unsure of. Should we revisit these semantics for just static private fields and methods, but leave static private methods actually on the instance? What would we do with the receiver--just pass it as the receiver of the function, and entirely ignore it for static private fields?

[bakkot]

@domenic, strictly speaking you can still do what you want without requiring all that overhead:

export const registry = new JSDOMRegistry();

let finalizeFactoryCreated;
export class JSDOM {
  #createdBy;

  #registerWithRegistry() {
    // ... elided ...
  }

  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);

    const body = await getBodyFromURL(url);
    return finalizeFactoryCreated(new JSDOM(body, options), "fromURL");
  }

  static fromFile(filename, options = {}) {
    normalizeOptions(options);

    const body = await getBodyFromFilename(filename);
    return finalizeFactoryCreated(new JSDOM(body, options), "fromFile");
  }

  static finalizeFactoryCreated(jsdom, factoryName) {
    jsdom.#createdBy = factoryName;
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}
finalizeFactoryCreated = JSDOM.finalizeFactoryCreated;
delete JSDOM.finalizeFactoryCreated;

But I agree this is significantly less clean.

On the other hand, the pattern that I've given works even if you're using fromURL on a subclass of JSDOM, whereas the static private solution does not, and which I think people will expect to work.

Edit: sorry, that that bit is false of course, since you've written JSDOM rather than this. Still, I don't know how well we'll be able to convince people not to use this, given that it is the overwhelmingly common way of writing it in languages Java, in my experience.

Edit 2: the second part of the edit was also false. :| Note to self: I should not try to remember how languages work when running a fever.

None of our options seem all that great.

[domenic]

(Replying to the edit.)

I'd be very surprised if people used this instead of the class name. I didn't even know you could do that in Java; when I was taught static methods it was always either unqualified or using the class name. At least one StackOverflow answer seems to back me up there.

[bakkot]

Now that I look, in fact you can't. Don't know what I was thinking, sorry.

[littledan]

@bakkot I don't think we should be thinking about delete as a viable option of a replacement. Without that, do you have other thoughts on whether private static methods are significantly motivated?

[littledan]

I've updated the explainer to advocate for the original solution, which does include static private methods, based on the feedback in this thread that they are important. Please keep the feedback coming!

[bakkot]

@littledan I expect that pattern could be captured by a decorator without requiring any deletes, but it would still be awkward, yes.

Without that, do you have other thoughts on whether private static methods are significantly motivated?

This issue gives a good motivation, I think, but I'm still worried about the subclass footgun. I think it actually is pretty likely to come up in real code if we go with the current semantics.

I'll riff off @domenic's example above (the 'code I want to write' option). Suppose that we the implementors of the JSDOM class furthermore decide we'd like consumers to be able to customize the behavior of the class further, say by providing their own implementations of getBodyFromURL, fromFile, or instance methods. Here's how I'd want to write that:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;

  #registerWithRegistry() {
    // ... elided ...
  }

  async static getBodyFromURL(url) {
    // ... elided ...
  }

  async static getBodyFromFilename(filename) {
    // ... elided ...
  }

  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);

    const body = await this.getBodyFromURL(url);
    return this.#finalizeFactoryCreated(new this(body, options), "fromURL");
  }

  static fromFile(filename, options = {}) {
    normalizeOptions(options);

    const body = await this.getBodyFromFilename(filename);
    return this.#finalizeFactoryCreated(new this(body, options), "fromFile");
  }

  static #finalizeFactoryCreated(jsdom, factoryName) {
    jsdom.#createdBy = factoryName;
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}

// elsewhere:

import { JSDOM } from "jsdom";

class SpecializedJSDOM extends JSDOM {
  static getBodyFromFilename(filename) {
    // ... some different implementation ...
  }

  static fromFile(filename, options = { something }) {
    return super.fromFile(filename, options);
  }

  serialize() {
    // ... some different implementation ...
  }
}

(diff)

```patch --- domenic.js +++ me.js @@ -6,20 +6,28 @@ #registerWithRegistry() { // ... elided ... } + + async static getBodyFromURL(url) { + // ... elided ... + } + + async static getBodyFromFilename(filename) { + // ... elided ... + } async static fromURL(url, options = {}) { normalizeFromURLOptions(options); normalizeOptions(options); - const body = await getBodyFromURL(url); - return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromURL"); + const body = await this.getBodyFromURL(url); + return this.#finalizeFactoryCreated(new this(body, options), "fromURL"); } static fromFile(filename, options = {}) { normalizeOptions(options); - const body = await getBodyFromFilename(filename); - return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromFile"); + const body = await this.getBodyFromFilename(filename); + return this.#finalizeFactoryCreated(new this(body, options), "fromFile"); } static #finalizeFactoryCreated(jsdom, factoryName) { @@ -28,3 +36,24 @@ return jsdom; } } ```

Unfortunately, this doesn't work: trying to call SpecializedJSDOM.fromFile('file') throws an error at this.#finalizeFactoryCreated because this is SpecializedJSDOM, which lacks the #finalizeFactoryCreated private slot. It could be made to work by using JSDOM.#finalizeFactoryCreated instead of this.#finalizeFactoryCreated, but since the preceding line needs to be this.getBodyFromURL (if it hardcodes JSDOM here, subclasses can't overwrite that method), I think it's going to be very easy to use the this form for the static private method as well as the static public method. Moreover, it feels really awkward to me to be forced to avoid it.

So while it's true that, as the readme explains, that linters and good error messages and so on might be able to discourage the use of this.#staticMethod(), I think it would be better if they didn't have to.

[littledan]

As an alternative @bakkot and I have discussed installing private static methods on subclasses, but omitting private static fields. Do you all have any thoughts on that approach?

[bakkot]

To add on to the above: come to think of it, there's a case where using JSDOM.#finalizeFactoryCreated just isn't good enough. (In fact @erights pointed this out a while ago.)

Suppose that #finalizeFactoryCreated wants to provide an extension point for subclasses, such as

export class JSDOM {
  // ...

  static #finalizeFactoryCreated(jsdom, factoryName) {
    if (typeof this.finalize === 'function') {
      this.finalize(jsdom, factoryName);
    }
    // ...
  }
}

If you can use this.#finalizeFactoryCreated, everything works out: the receiver for #finalizeFactoryCreated is the subclass, and so the implementation of #finalizeFactoryCreated can see the subclass's this.finalize. If you're forced to write JSDOM.#finalizeFactoryCreated, that simply doesn't work and cannot be made to work.

---

This sort of thing is why I am in favor of the approach @littledan mentions.

[domenic]

I am very skeptical of this hypothesizing about desiring some sort of polymorphic dispatch on static members, and especially private static members. I don't believe any other language supports that, and I don't think we should be adding machinery to JS (such as copying the methods to subclasses) in order to support that use case.

My mental model of statics is essentially that they are lexical declarations at class level. I realize for public static you can do polymorphic dispatch, but I am fine saying that is not true for private statics, especially given that private instance fields are already very anti-polymorphic by virtue of not participating in the prototype chain.

[littledan]

@domenic I'm not so convinced by the "code grouping" requirement. There's lots of things that we don't allow grouped, e.g., class declarations in the top level of a class (you could do this through static private fields but it would look weird). I think it should be OK to put things that are not exported outside the class declaration; I don't think initializer expressions are likely to contain something that needs to reference private fields or methods.

One reason that supporting private static methods on subclasses isn't all that ad-hoc is that it's actually sort of analogous to instance private methods--it's installed on all the objects that it would've otherwise be found on the prototype chain. Does this make @bakkot 's proposal any less excessive-feeling to you?

About polymorphic dispatch on static members: in another thread (having trouble finding it now), @allenwb explained that ES6's practice of assembling a prototype chain of constructors is deliberate, useful and matches other dynamic object-oriented languages. About the complexity of private fields in JS and the implications of that: It's true that this is relatively new ground, largely because other dynamic languages do not attempt to enforce privacy to this extent (but we have good reasons for making the choice we did).

If the mental model of statics is that they're lexical function declarations, then this would allow for the genericity in @bakkot 's comment. If you really want lexical semantics, we could just straight-up say that private static fields are just lexically scoped variables and private static methods are just lexically scoped functions (all with funny method/field-like syntax). Would anyone prefer that?

[bakkot]

@littledan

About polymorphic dispatch on static members: in another thread (having trouble finding it now)

I'd guessing you're thinking of this comment.

[littledan]

tldr: I'll change the semantics to be, you can use this.#foo() on subclasses for static private methods, but for static private fields, you have to use ClassName.#bar or otherwise risk getting a TypeError

Following some discussion on #tc39 with @bakkot, @domenic and @ljharb , I'm leaning towards a new intermediate option: Private static methods are installed on subclasses, but private static fields are not and still have a TypeError when read or written to subclasses. I'll edit the explainer to give more detailed reasoning and write spec text specifying this option, but to summarize:

• Adding private static methods to subclasses will be useful and also follow similar logic to instance private methods: although these methods are specified as own fields of the instance/constructor, they are logically as if they are shared based on the prototype chain, since they are immutable and set on each object that would have been able to access the method through the prototype chain.
• For private static fields, because they are mutable and may be shadowed in the prototype chain, there's no good, analogous semantics for them to have in subclasses. It would be fairly crazy to replicate the semantics of ordinary properties and prototype chains, and partial workarounds (e.g., allowing reads but not writes from subclasses) are very complicated. These would be included but throw a TypeError when read from or written to in a subclass.

[wycats]

@littledan I quite like this outcome.

It aligns class initialization more closely with instance initialization where it can be done without confusing side effects, and disallows cases where side-effects make all of the options compromised.

At the same time, the use-cases for static private fields can virtually always be accomplished using a lexical variable, which was not the case for static private methods.

TLDR: Between really needing static private methods semantically and the fact that they aren't confusing due to side effects, there's a strong argument for making them work. Inversely, between not really needing static private fields semantically and the fact that they are confusing due to side effects, there's a strong argument for disallowing them. I like this conclusion.

[ljharb]

@littledan to clarify, for static private methods you can use (where #foo is a static private method on a superclass) this.#foo() on subclasses, but not in them, correct?

[littledan]

@ljharb Exactly, thanks for the clarification. I fixed the wording as you suggested.

[bakkot]

@wycats To make sure we're on the same page, the proposal is not to disallow private static fields, but rather to allow them with the semantics that referring to this.#staticField within a static method invoked on a subclass would throw (even though this.#staticMethod() would not). Is that your interpretation, and if not is it still something you're on board with?

[wycats]

Ah I didn't understand that, so thanks for the clarification.

I think, for me, the consequence of this decision is that I would avoid static private fields and teach people to avoid them.

Was there a strong argument for static private fields that made us want to give them these semantics rather than defer them?

Either way, I could live with this conclusion.

[littledan]

The argument was given by Domenic above in https://github.com/tc39/proposal-static-class-features/issues/4#issuecomment-354184890 . My personal feeling about static private fields is that we could take them or leave them, but if we take them, they should probably have the semantics from the original proposal (unless we allow reads but not writes on subclasses, though this would be pretty complicated).

@wycats I'd like to understand why you'd teach people to avoid private static fields in this case. Do you think it'd be hard to teach people to use the class name when referring to private static fields? I wonder why your impression of this differs from @domenic 's.

[littledan]

Thinking about it a bit more, I think static private fields will be useful for similar reasons to static private methods--you may want a place that you can reference instance private fields deep within some object which is created just once per instantiation. Here's an example (though it's in the context of a possible future operator overloading proposal) https://gist.github.com/littledan/19c09a09d2afe7558cdfd6fdae18f956

[littledan]

It seems like we're at a bit of a contradiction in terms of goals for static private fields. Between this thread and other conversations with TC39 members, I've heard about the following goals:

1. Static private fields should exist (e.g., @domenic above)
2. If private static fields exist, it should be possible to refer to them from a static method as this.#field (not always ClassName.#field), with semantics that are generally similar to how static public fields work (e.g, @rbuckton)
3. Private access shouldn't be interceptible by Proxies (e.g., @erights, was a big topic of discussion in the ES6 cycle; if private static field access traversed the ordinary, mutable-by-default prototype chain, an "attacker" may be able to insert an element in the chain and change a subclass's observed value)
4. A second parallel "private prototype chain" (as in #7) would be complicated and undesirable (my assertion)

I don't see a way to satisfy all of these constraints at once. Does anyone else? Which one(s) of these constraints is less important than the others?

[ljharb]

Could you provide or link to the rationale for the second point above?

[domenic]

(I suggest editing your posts to number the constraints.)

There's two constraints here that you haven't quite captured, perhaps because together they amount to your number (2), so they don't quite fit into the hierarchy. But what I heard was:

• (2.1) Private static methods must be methods, callable as this.#method() not just ClassName.#method(), with semantics that are generally similar to how public static methods work. (e.g. @bakkot)
• (2.2) Private static fields and private static methods must behave the same with regard to this vs. ClassName usage. (e.g. @adamklein, @msaboff, @kmiller68)

To me, the constraint ordering is (4) < (2.1) < (2.2) < (1) < (3).

[ljharb]

Thanks, that clarifies it.

My ordering is thus 2.2 < 4 < 1 < 2.1 < 3 (with 3 as the most important, and 2.2 as the least)

[littledan]

@ljharb I think the main (2) was another thing considered directly desirable, though perhaps not an absolute constraint, e.g., see the positive responses in https://github.com/tc39/proposal-static-class-features/pull/7 and the proposals by @rbuckton (1, 2) . Some 1:1 discussions I've had with @jridgewell have come back to this point as well, but nothing that I know of that's published somewhere.

@domenic Done, thanks for the suggestion and for these additional points.

If others want to respond with their own ranking, these are very useful data points. Please keep it coming. cc @erights @gsathya @xanlpz @erights @waldemarhorwat @zkat @bakkot @ajklein @kmiller68 @msaboff @allenwb @bterlson @rbuckton @bmeck @wycats @rpalmer57 @tschneidereit

[gibson042]

I'm pretty sure that more than one approach is capable of satisfying all of those goals, possibly with the exception of (4). For example:

• Every class has a [[PrivateFields]] weak map—keys are classes equal to or descended from the class¹, values are maps with a key-value pair for each private field defined by the class.
• target.#field accesses use target as a lookup key for the [[PrivateFields]] map associated with the closest containing class of the source text in which the access appears. Optionally, they may use target's [[ProxyTarget]] as appropriate. Failure of such lookups to retrieve data permitting the access results in an exception (I'm not sure if ReferenceError would be appropriate, but if not that then TypeError).
• Changing the class hierarchy (definitely when creating a subclass, and perhaps also when changing the prototype chain of an existing class²) affects the [[PrivateFields]] entries at every level up from the class gaining (or losing) a parent. To prevent interception by proxies, independent [[Prototype]] tracking may be required (but depending on precise semantics, this may violate goal (4)).

---

¹ Or at least descended from it at one point in time (exact semantics left unspecified here). ² Also unspecified here, though it could clearly strain goal (4) in cases where the class being reparented has child classes of its own unless [[PrivateFields]] is made enumerable.

[littledan]

@gibson042 That's a plausible idea, but the semantics would differ in some cases, for example:

class Grandparent {
  static #count = 0;
  static inc() { return this.#count++; }
}
class Parent extends Grandparent { }
class Child extends Parent { }
Parent.inc();
Child.inc();  // returns 0 or 1?

[gibson042]

The approach I provided above is just a general description of moving parts, not the particulars of how to choose specific behavior from many alternatives (but I did try to make such options explicit in #5).

As noted in #7, though, I dislike both copy-on-write (cf. https://github.com/tc39/proposal-static-class-features/pull/7#issuecomment-355847956 ) and single-binding (cf. https://github.com/tc39/proposal-static-class-features/pull/7#issuecomment-355795301 ) semantics for static fields. My preference would be for the first Child.inc() to return 0. In the above approach, that would be specified in the third bullet point as something like "when setting the parent of a class, an entry is created for it in the [[PrivateFields]] map of each new ancestor class, with the value for each field initialized to {if yes to Question 1a1: the current value for the ancestor, if no to Question 1a1: the initial value}" (as opposed to initializing to unset for "copy-on-write" or values being irrelevant for "single-binding").

I guess the questions could also be recast as something like

• How many bindings for each static field?
  • 1 per class
  • 1 per class hierarchy
• When is a static field binding created?
  • on class initialization
  • on class initialization or ancestry manipulation
  • on write
• To what value is a static field binding initialized?
  • initial value for lexical class
  • current value for lexical class
  • current value for parent
  • reevaluation of initial assignment expression
• What happens to a static field binding when an ancestor class is abandoned?
  • it remains accessible (for e.g. oldParent.method.call(traitorClass))
  • it remains but is suppressed (reconnecting to the ancestor will restore the values)
  • it is removed (reconnecting to the ancestor will not restore the values)

[littledan]

@gibson042 These semantics are interesting. I believe with the current decorators proposal, you'd be able to write a decorator to implement them. Would this help? I'm a bit worried about the increase in the size of the mental model if we introduce these semantics for just static public and private fields, and not all prototype chains (because we can't change existing prototype chains).

[allenwb]

A very interesting thread. The use case that @domenic identified is simple procedural decomposition. It should always be possible and simple. Unfortunately the direction that ES "private" is heading seems to make it difficult and conceptually confusing. I think the reason is that the proposed designs incorporate a mishmash of procedural and OO gadgets (along with some Java cargo-cult misunderstandings) that lead to language design hacks and seemingly arbitrary special cases semantics as we try to ensure that the design supports various use cases. With each design tweak the semantics becomes more and more complex. Instead we should be trying to simplify, simplify, simplify...

So, lets go back to the basics.

First, procedural decomposition is the process of designing (or refactoring) a (larger) procedure as calls to a set of smaller procedures that collectively produce the same result. Procedural decomposition is a basic abstraction mechanisms that is used in all modern programming. It helps us manage complexity by breaking large problems (procedures) into smaller, more understandable problems. It also provides the mechanism by which recurrent sub-problems can be solved once at a single place and then reused from many places.

Next, in JS we have basically two kinds¹ of procedure calls baked deep into the call semantics: lexically-resolved calls and object-resolved calls. As should be obvious from the names I choose, the primarily difference is how the actual invoked function is determined. But the two different call forms also have an impact on how programmers conceptualize and use procedural decomposition.

With a lexically-resolved call, the primary determinant of the invoked procedure is the static location of the call operation within the overall lexical structure of the source code. A programmer who codes a lexically-resolved call is thinking something like: "at this point in the program when I call 'foo' I'm invoking the function defined at line 213". This is classic procedural decomposition, a subproblem is solved by a separate distinct procedure that was selected when the code is written. We have statically decomposed a large procedure into a fixed set of smaller parts.

With an object-resolved call, the primary determinant of the invoked procedure is a runtime object value that qualifies the call. A programmer who codes an object-resolved call is thinking: "at this point in the program when I call 'obj.foo' I'm invoking the 'foo' function provided by 'obj'". What actually happens will depend upon the value of 'obj'". This is a different way to think about procedural decomposition, a subproblem may have many distinct solutions and the selection of an actual solution is dynamically delegated via an object value. In this form of decomposition of we defining an abstract algorithm, not a fixed decomposition of a single statically defined procedure.

Note that I'm emphasizing how a programmer conceptualizes what they are coding. These are two very different ways to think about what it means to make a procedure call. The significance of the distinction was deemed so important that Smalltalk, the first truly OO language, only supported object-resolved calls and coined new terminology, "message send", to emphasize how such calls differs from the then prevalent understanding of procedure calls.

Which brings us to a language design challenged. Can we design a language that provides direct support for both lexically-resolved calls and object-resolved calls but in a manner that reinforces rather than muddles the important conceptual differences between the two call forms and the two styles of procedural decomposition?

Note that this is mostly a challenge for multi-paradigm languages. Classic procedural languages (Pascal, C, etc.) either only support lexically-resolved calls or use distinctive syntax for their limited support and usage of object-resolved calls. Pure OO languages such as Smalltalk take the opposite tack and only support object-resolved calls (or have distinctive syntax for limited use of lexically-resolved calls).

Classic Java is interesting in that it appears to be a pure OO language (all procedures are defined as methods within class definitions) but it unobtrusively also has fairly extensive support for lexically-resolved calls. But that support is disguised so successfully as apparent object-resolved calls that many Java programmers don't even realize it exists as such or that they are using them. Where are they hiding? Behind the private keyword with an assist from Java's static type system. Every Java private method is a procedure that is always invoked via a lexically-resolved call. This doesn't create any confusion because Java doesn't have any other syntactically distinct way to define a lexically-resolved procedure. It suffices for a Java programmer to think about private methods as the degenerate case of object-resolved calls where there is only a single possible target for each call. But note, that this isn't the case for Java's protected methods -- Java protected methods are invoked using object-resolved calls. In Java, if you want to do classic static procedural decomposition of a methods you use private sub-methods. If you want to use OO decomposition to define an abstract algorithm but in a manner that limits who can implement/access the constituent sub-parts you use protected sub-methods.

JavaScript however is different. It provides distinct syntaxes to support the definition (via function definitions and arrow functions) of procedures that are intended to be invoked via lexically-resolved calls and the definition (via concise method definitions) of procedures that are intended to be invoked via object-resolved calls. Part of the design heritage is that a strict encapsulation mechanism is available for the syntactic forms used to define the targets of lexically-resolved calls but only weak encapsulation mechanisms are available for the targets of objet-resolved calls (ie, nothing equivalent to protected).

Thus we encounter @domenic's use cases and the language design challenges they presents. What Domenic is doing appears to be classic procedural decomposition. He is not trying to define an abstract algorithm whose sub-steps can be individually defined/over-ridden using object-resolved calls (if that was what he wanted to he could accomplish it using static methods). As he showed, in some cases he could accomplish the procedural decomposition using function declarations that are outside of his class definition. But, some use cases require that those function have access to language features that must only be available to code that is lexically within a class body. What is the simplest solution? Why not simply allow those function declarations to appear within class bodies. Then they are encapsulated by the class declaration, lexically-resolved calls (callable from both instance and static methods), that have full access to private state and other lexically constrained class gadgets. But they aren't object-resolved functions and will have minimal conceptual overlap with OO design patterns.

To rap up, I'm really disappointed in the complex mess that ES class definition seem to be turning into. We need to simplify, simplify, simplify!

My ideal design simplification would only have:

1) ES2015 class features (public prototype and constructor method properties) 2) Per instance and constructor public data property definitions with initializers. 3) "private" #slots on instance objects (always accessed via an object qualifier, no naked #foo access there-by maintaining a syntactic distinction between lexically resolved and object-resolved accesses) 4) Allow function, let, and const lexical declarations within and scoped to class bodies.

This covers all the essentials while clearly distinguishing object-resolved and lexically-resolved calls and accesses. I know this isn't the direction the committee is going. I suspect we will live to regret it. \<\rant>

¹ We also have higher-order usages of function values, eg. function valued parameters, variables, function results. These are typically lexically scoped to a binding that may have a value that must be dynamically determined. For this discussion, I'm treating them as lexically-resolved calls even though the actual target function but be dynamically determined at runtime.

[bakkot]

@allenwb, that's a really helpful summary, thanks.

As I understand it, the problem we've always run into with lexical declarations in class bodies is that it seems like programmers expect the natural syntax there to mean instance properties or somehow otherwise be per-instance. This seems to be true no matter what the actual syntax for instance properties is.

Yes, I know, that's a weird expectation from a language design perspective, but it appears to be the world we live in. That's always been my objection to lexical declarations in class bodies: for all their elegance from a language design perspective, I think they would be really confusing for users.

(Though I guess it is the case that function declarations, uniquely, do not have this problem. Still, given that we already have method syntax in class bodies, I think the syntax for function declarations in class bodies would end up pretty weird:

class Foo {
  function m(){} // a lexical declaration
  static m(){} // a static method
  m(){} // an instance method
}

Still, I guess I could live with that.)

Otherwise, I think the committee is going in the direction you want: that is, we seem to have pretty strong consensus on points 1-3, I guess with the caveat that some people have suggested later reintroducing the shorthand syntax, and we're trying to figure out how to accomplish what your 4 accomplishes while still having syntax we can live with.

[gibson042]

FWIW, I agree with @allenwb's last point—classes are already too complex, and aren't getting any simpler. Static methods are like the camel's nose under the tent ("we shouldn't have static methods without static fields… of course static fields are expected to be mutable… and they really should make sense in subclasses too… and I guess we should also have instance fields…"), and #-slots bring in tons of edge cases. In fact, my primary goal with the comments and issues here is making sure that they are addressed explicitly rather than sneaking in.

But to be honest, I'd much rather see the #-slots left out forever in favor of class body lexical declarations as suggested, because those don't carry any baggage with respect to hierarchy updates or unexpected method receivers. Leveraging closures for privacy instead of new syntax is more consistent with the history of Javascript anyway.

Pursuing that path would leave class bodies consisting of instance methods, static methods, and unexposed let/const lexical declarations and function declarations. And if a sufficient case code be made (and edge cases sufficiently addressed), maybe also instance fields and static fields (though I'd be inclined to leave them out and just let things simmer).

[wycats]

@allenwb @gibson042 before I try to respond to the latest comments, I want to figure out whether we're losing coherence on the overall shape of the proposal.

I (personally) would consider losing static-private-everything a variant of the current Stage 3 proposal. I would not consider replacing all private state with lexical variables and private methods with lexical functions to be a variant of the current proposal.

I don't yet have an opinion about whether I have a problem with losing coherence, I just want to understand where we're at before writing a response 😄

[gibson042]

My own position is one of uncertainty, especially about static fields (both public and private), and I'd oppose introducing them without strong consensus on those questions I shared (which I don't even have inside my own head right now). But the same goes for instance private fields as well, so I guess that really is "losing static-private-everything" from https://github.com/tc39/proposal-class-fields .

Private lexical declarations do seem cleaner to me, but I agree that such a shift would be too dramatic to be considered a refinement of this proposal. It's no surprise to me that process work is likely more difficult than technical work, though. "Losing coherence" seems about right from my perspective—not "lost", but headed in that direction. Better to exercise excessive caution than to saddle the language with a minefield.

[bakkot]

I think the committee - including Allen above, if I'm reading him right - has consensus on public and private instance fields in their current form, and I am not aware of any remaining open questions about their motivation or semantics. (Also, they're starting to ship!)

I don't think we ought to revisit those in this thread. If you want to think about it that way, I would add to @littledan's list of constraints above an implicit and at least for me mandatory (0): that public and private instance fields go forward with their current syntax and semantics. They're a much more important feature than static anything, and I think we have the right design for them; I would not want to - or to be precise, would almost certainly not be willing to - compromise on that design to serve the needs of static features, unless there is some natural design we've somehow overlooked.

[allenwb]

@bakkot

As I understand it, the problem we've always run into with lexical declarations in class bodies is that it seems like programmers expect the natural syntax there to mean instance properties or somehow otherwise be per-instance.

Is this conclusion data based? Who was survived?

Even if it is, should it be the determining factor? It seems to me that much of the difficulty we've had with extending class declaration functionality has been dealing with expectations that programmers (and us) may carry over from other languages. It seems that we have primarily made progress and avoid complexity when we focus on what makes sense in the context JS rather than worry too much about what various programmers may have learned in other languages. Ultimately they have to lean JS and will appreciate a simpler, more internally consistent language.

(Though I guess it is the case that function declarations, uniquely, do not have this problem.

I included let and const primarily for completeness so we could say that any of the lexically-scoped declarations could occur in a class body and have the same meaning they would have if they occur in a block. Also, let would is a simpler way to address most of the use cases people may have for "static private fields". But, they aren't essential to supporting @domenic's procedural decomposition use-cases. If let/const were not allowed within class bodies the same semantics can be obtained by placing a let/const lexically outside of the class declaration. This works for them (and not for function) because the initializers of let/const declarations would rarely, if ever, need to access #fields defined within the class body.

So, arguably only allowing function is a further simplification.

Still, given that we already have method syntax in class bodies, I think the syntax for function declarations in class bodies would end up pretty weird:

My whole point was that there are important differences between "methods" (object.resolved calls) and lexically-resolved functions and that it is problematic for the class body design to blur that distinction. If we are going to have lexically-resolved functions in class bodies (and @domenic showed why we need them) then the least weird thing is to use the syntax (function declarations) the language already provides for exactly such functions. What does function mean is a class body? It means exactly what it means anywhere else, it defines a lexical function binding that is visible to everything within the scope of the closest surrounding { }.

[wycats]

@bakkot I agree.

My impression is that instance fields (public and private) and private instance methods and accessors have good consensus.

I have also heard strong and persuasive arguments from @ljharb that public static fields reflect popular user-space patterns and should be supported. Because public static fields are normal properties, a lot of the questions raised in this thread are less relevant (and any strangeness could be explained without creating brand new mechanisms).

On the other hand, private static fields, methods and accessors have been the subject of more debate. Their semantics overlap strongly with the existing use of lexical scope and new mechanisms are needed to explain their behavior (or risk very serious footguns that the committee seemed unwilling to accept).

The main reason that private static elements are attractive is that they offer a way to share private instance state with static state. In the absence of that requirement, lexical variables outside the class would serve perfectly (and I would claim, would be more intuitive and in line with existing patterns).

This suggests to me that we ought to explore other solutions for allowing lexical functions to have the private names in scope. Allen's proposal for class-scoped function declarations is one such proposal. Declared private names outside of a class body is another.

But I don't agree with the idea that lacking static private elements would indicate a "piecemeal" approach to the design. Static private elements are different enough from instance private elements that we ought to be able to consider them separately, along with other proposals that might make them unnecessary.

But I strongly disagree with revisiting instance-private features at this stage.

[allenwb]

WRT "coherence", Here is where I'm at.

1) static private methods and fields in the forms that have been discussed are highly problematic , essentially the straw that breaks the camels back.

2) But, @domenic has demonstrated that there is a real need to support procedural decomposition within class bodies that includes the ability to use private field references.

2) there are no issues with "static public fields". They are just initialized data properties of a constructor. Not really essential, but useful when a class wants to expose constants or other data values as constructor properties.

2) No issues with private and public instance "fields". (Well, I hate the keywordless syntax, but that isn't a fatal issue).

2) private instance methods are problematic because they violate the lexically-resolved/object-resolved distinction I described. They syntactically manifested as if they were object-resolved methods when they are really lexically-resolved functions. They start us on the slippery-slope of complexity and confusion.

So, I'd be relatively happy if we rolled out just public/private instance fields and won't have a problem if we also included public static "fields".

But that doesn't address @domenic's procedural decomposition issues and I think that is going to be a significant issues so we would still have an incomplete solution. I think if we include class body lexical 'function` declaration we will have covered all the key class capabilities and won't have to do additional work (except for decorators) on class definitions for a long while.