Non-debate thread: explaining the reasons for this proposal's decisions

[mbrowne]

The goal of this thread is to begin consolidating and documenting the motivations and considerations that led to the current proposal. I think this information will be very helpful for explaining the proposal to newcomers to help mitigate negative or perplexed reactions and move them more toward, "ah, that makes sense." I think it's also useful to document the reasoning behind decisions for posterity, so that anyone interested in deeply learning the language can understand how it evolved and why. (This includes why it was chosen over alternative proposals.)

Please pay attention to the title of this thread: I would like to avoid debates here (just in this thread). I am very much in favor of healthy debates (and have participated in plenty here) but I think for the purpose of this thread they would be noisy and distracting.

Let me start by summarizing the general considerations regarding hard privacy. From what I can gather from @ljharb's comments, first of all hard privacy is seen as a very important requirement because there are use cases that can't be achieved any other way. Some of those use cases, such as making it impossible to branch off of the existence of a private field, or avoiding collisions with public fields, are particularly important to many library authors. It also sounds like there are members of the committee who believe that "privacy" in other OOP languages doesn't go far enough and that hard private is simply a better default—a more correct way of ensuring encapsulation.

Is that a fair summary so far of the motivation for hard private fields? I think these are good and valid arguments, but I think the case would definitely be stronger if it can be established that alternative options for hard privacy such as private symbols are unacceptable—especially given how consequential this decision is.

So I would like to pick up from my comment in https://github.com/tc39/proposal-class-fields/issues/175#issuecomment-441646222:

It would be possible to use the private symbols proposal to cover the use cases where hard private is absolutely needed, and let private fields be soft private. I'm wondering, why did [the committee] reject this option?

It seems there is an unresolved question related to this from @Igmat: https://github.com/tc39/proposal-class-fields/issues/175#issuecomment-442047721 https://github.com/tc39/proposal-class-fields/issues/149#issuecomment-441057730

This proxy issue with private symbols seems like a key point...after all, even @ljharb said:

private symbols, i believe, would be hard private, not soft, and if so that’d be fine with me - but I’m told they have issues with membranes, and as such are not an option.

OTOH, it's moot if the committee's position is that soft private is simply a bad default even if there are other ways hard privacy could be achieved. I'm not clear on the committee's reasoning on this.

[mbrowne]

I forgot to mention...for those who haven't checked the readme for this repo in a while, more details on committee decisions can be found in the links provided under "Development History": https://github.com/tc39/proposal-class-fields#development-history

So this thread is for clarifying and asking follow-up questions, not for revisiting the whole decision process from the beginning.

[littledan]

The argument against private symbols eventually became, once you have all the "right" semantics, it is so far away from ordinary properties that it's not reasonable to make them treated as property keys with [] anymore. That's why decorators makes PrivateName not a Symbol/property key, even though it serves the same role.

[littledan]

Apologies for my delay here and thanks for following up like this!

[Igmat]

once you have all the "right" semantics

@littledan what are the "right" semantics?

[littledan]

@Igmat, @erights has repeatedly stated that the semantics must be analogous to WeakMap, so that's what I was getting at with "right". I see this as a little more of a tradeoff than an absolute (but, in the end, a decent tradeoff), which is why I used quotes :)

[mbrowne]

Thanks @littledan. Can you comment on the committee's overall views regarding alternative solutions for hard private? For example, if private symbols didn't have these issues and were seen as a fully workable proposal, would the committee have been likely to consider making private fields soft private instead of hard private? Or would it have maintained its preference for hard private class fields? I'm asking these hypothetical questions as a means to an end: I'm trying to get at the driving forces behind the total rejection of private x syntax. I'm piecing it together, but with so many different arguments that have been presented, it's hard to tell from the outside what really sealed the deal and how the wishes of various stakeholders (e.g. library authors, users interested in reflection features, etc.) were balanced.

FWIW I see both pros and cons of fields being hard private by default, and even without knowing all the details the committee's decision on this seems very reasonable to me. So I'm not asking for a primer on the benefits of strong encapsulation by default, but rather clarity about which of my points in the OP were most salient to the committee and whether I missed anything that was a significant factor in the decision.

[bakkot]

For example, if private symbols didn't have these issues and were seen as a fully workable proposal, would the committee have been likely to consider making private fields soft private instead of hard private?

I don't think so. You can read some of the discussion here, if you like, but my position and, as I understand it, the position of several other members of the committee is that "soft" private is not meaningfully distinct from symbols as they currently stand, so if we're adding a new thing for private state, it should actually provide privacy.

As to which should be the default, it is my position that it should be the case that the default behavior for a new feature is the new capability it provides, not the thing you could do already.

(That said, private symbols would be "hard" private, as I use the term, unless they triggered proxy traps.)

I'm trying to get at the driving forces behind the total rejection of private x syntax

The FAQ already summarizes my position on this to the best of my ability. The short version:

• It is unacceptable for adding private state to an object to prevent other code from adding any public property to that object (or its own private state, no matter the name).
• It is unacceptable to introduce static types as a prerequisite for this feature.
• It is unacceptable for a single occurrence of this.x to mean different things at runtime depending on the value that this happens to have at the time.
• It is unacceptable for introducing or renaming a private field to change the behavior of code elsewhere in the class which is not intending to operate on instances of the class.
• As a consequence of the previous four points, having access be this.x is not workable.
• It is unacceptable to have field declarations of the form private x without having this.x access that field.
• As a consequence of the previous two points, private x is not workable.

None of these points really depend on the hard vs soft thing.

Of course, other committee members may have their own opinions. This is an inevitable part of the consensus process - even if everyone agrees on a design, they won't necessarily (or typically) agree on the reasons for it. So I'm afraid you're unlikely to get a statement from the committee as a whole on this.

(Edit: forgot one more point: It is unacceptable for the this keyword to get any new special semantics.)

[littledan]

My understanding of the committee's position matches @bakkot's, even if my personal feeling is that many of these things are tradeoffs rather than unacceptable.

[panlina]

This is a good suggestion. Can anyone pick up to summarize considerations behind [[Define]] semantics for public fields?

[mbrowne]

Thanks @panlina...I was intending to follow up on this thread, just hadn't gotten to it yet.

First of all, thanks very much to @bakkot and @littledan for the explanations. I think we have the beginnings of a rationale document here. I think there is also another point in favor of hard privacy in addition to what you mentioned, which is that soft privacy (without making fields fully public) is still easily available via decorators. Although...it's not actually quite as easy as I originally thought it would be, since only field-level decorators (and not class-level decorators) have access to private fields in the latest decorators spec, but that's a discussion for the decorators repo.

[mbrowne]

Regarding [[Define]] vs. [[Set]], I agree that's a very good choice for a next topic to discuss here. I still don't really understand the case for Define and strongly favor Set (for practical reasons), but I'll take a stab at summarizing the argument to get the process started...

As best I can tell, the main argument in favor of Define is avoiding side effects, meaning that declaring a public field should always do one and only one thing: define an own property of the instance. It seems that the distinction between declarative and imperative syntax is important here; field declarations are intended as declarative syntax, so side effects such as a getter or setter being called (which would be acceptable in an imperative setting) are undesirable. What I don't think has been demonstrated is that there are significant practical downsides to Set, or at least none that come close to the significance of the practical downsides of Define, i.e. potentially confusing issues for developers and resulting bugs. (Purely declarative semantics do have the practical benefit of helping to make code more readable and predictable in general ways, but I'm referring more to specific points of confusion or surprising behavior specific to Javascript and its established expectations.) I bring this up here because although this is a non-debate thread, I would definitely appreciate any committee members describing more practical issues with Set that I might have missed.

One other issue with Set I've seen mentioned by committee members is that it could complicate the implementation of decorators. @ljharb offered an alternative interpretation, saying that Define also comes with its own challenges for implementing decorators, and it's more of a 6 in one half dozen in the other situation. @littledan, perhaps you could weigh in here as one of the champions of the decorators proposal (and also this proposal of course)?

[mbrowne]

I followed up on my point about reflection here: https://github.com/tc39/proposal-decorators/issues/191

[littledan]

https://github.com/tc39/proposal-class-fields/pull/196 takes a first stab here. I merged it already, but review comments still welcome.

[mbrowne]

Thanks @littledan. In addition to what you wrote in the readme, would you say that the points I mentioned above (declarative syntax, avoiding side effects, and possibly working better with decorators) were the main reasons for this decision?

Also, I am wondering about @rdking's suggestion (which I think @ljharb said he had proposed as well) to use := for Define and = for Set, so developers would be able to use either without having to use decorators. (Personally I'm not sure that introducing a new operator for this is a good idea, but my reasons might be different from the committee's, so I'd like to hear your thoughts on this.)

[bakkot]

@mbrowne, I think the committee has previously discussed and rejected having both, for roughly the same reason it has discussed and rejected having syntax for both hard and soft private - it means another thing to think about every time you write a field, which will often not make much difference. (Here more than with hard vs soft private - set and define semantics will be actually identical a large majority of the time.)

[littledan]

@mbrowne Yes, those are all important design considerations as well.

[mbrowne]

I just wanted to note that I welcome involvement from anyone else who wants to work on further documenting the rationale for this proposal. I might return to this at some point, but I think the readme and FAQ are more complete now, and I no longer have many questions that are really perplexing to me personally--I mostly get the reasoning now even in the (few) areas of this proposal where I have doubts or disagree. Although I'm not going to personally pursue this at the moment, I would say a next area that could be better summarized/consolidated for posterity would be the reasons this proposal is considered superior to classes 1.1, class members, private symbols, etc.

[littledan]

Feel free to mine content for this text from slide decks I wrote in response to those ideas in the TC39 meetings where they were/will be presented, linked from the agendas.

[mbrowne]

There's something about this proposal and the surrounding discussions that I still don't understand, which is the attitude that native protected fields will probably never be added. It can't just be that JS is a dynamic language, because Ruby is dynamic and it has protected members. As long as protected fields in JS were prefixed with # (and perhaps declared as protected #x), it would be easy for the engine to disambiguate from public properties, which would lessen the performance cost of adding this feature. As a reminder, I'm not trying to engage in any debates in this thread...I'm sure there are very good reasons for the aversion to a native protected feature. I realize they are also quite technical, but if it's possible to explain without having to actually read and understand all the source code of a JS engine, I would be very curious to know.

[littledan]

It's not about performance, but about the definition: how would we reliably distinguish "inside" from "outside" the class when subclasses are considered "inside"? I can't think of a strong way.

[mbrowne]

The parent class has always been evaluated already at the time the subclass is being evaluated, right? Protected fields could work according to different rules than private ones. I'm under the impression that there's a practical barrier here but I'm not understanding what the real issue is, i.e. why does it matter that subclasses are syntactically "outside"...is the mutable [[Prototype]] slot part of the issue here?

[littledan]

The problem is that you could make a subclass which reads the protected information, and then .call() this method on some other instance, so there is really no strong protection.

[mbrowne]

So are you saying that it would be doable if it were not for the existence of.call and .apply in JS? (Well, I suppose the entire mechanism of calling methods on objects could be modified so that special exceptions could be made for call/apply and protected fields, but I imagine that would be impractical.)

[nicolo-ribaudo]

No, you would still be able to do this:

class A {
  protected #x;
}

class B extends A {
  constructor(secret) {
    this.#x = secret;
  }
}

class Thief extends A {
  static steal() {
    return this.#x
  }
}

var secure = new B("secret");

secure.steal = Thief.steal;
var secret = secure.steal();
delete secure.steal;

[mbrowne]

@nicolo-ribaudo Can't you do that in any of the existing languages with protected members? The way I see it, protected access is about preventing naiive or inadvertent access outside a class hierarchy; it's not meant to be a security feature.

But I have a separate question, while you're here. Suppose someone didn't want to use TypeScript for whatever reason, but they wanted protected fields and methods. I'm just curious, would it be possible to write a Babel plugin for protected members without completely rearchitecting Babel? Obviously it would require more static analysis than the average Babel plugin...

[ljharb]

Symbols already prevent naive or inadvertent access outside a class hierarchy.

[mbrowne]

Good point. Of course symbols make fields more verbose to declare. And they prevent the use of dot-notation, but I guess that's not a big deal. But it would certainly be easier to write a Babel plugin to convert a field name to a symbol than to actually enforce that a field is not being accessed from an unrelated class (or externally). And hopefully in the future we would be able to do that with a decorator too.

[littledan]

I think the next step for this thread would be to make a PR (or a series of them) to add the relevant clarifications to the readme or FAQ. I am not sure what clarifications would be beneficial, since I think we already have pretty good coverage of these points, but I would welcome PRs to cover my blind spots here. We've covered all of these points at length in other threads; it's the time to consolidate the documentation now.

[mbrowne]

@littledan I'm not aware of a thread where the technical challenges and potential problems with a native protected feature were discussed in detail. Do you have a link? (Only if it's easy to find; I wouldn't want you to go searching.) Thank you.

[nicolo-ribaudo]

I'm just curious, would it be possible to write a Babel plugin for protected members without completely rearchitecting Babel? Obviously it would require more static analysis than the average Babel plugin...

It depends on how much declarative they are. If they end up being something like this, it wouldn't be too hard:

class A {
  protected #x;
}

class B extends A {
  inherit #x;
}

[rdking]

@littledan If I can provide you with a strong means of implementing protected that does not risk leaking the scope to sibling classes, can consideration for protected be brought back to the table?

I've provided this link many times before. If you take the time to review it, I'm sure you'll see that it provides some potential for the strong disambiguation you want.

[mbrowne]

I think we should continue this discussion in a different thread. Admittedly my question about Babel doesn't really belong here either.