Symbol.private vs WeakMap semantics

[Igmat]

Originally I posted following messages in #149 answering to @erights concern about Membrane pattern implementation with Symbol.private approach. I've decided to create new issue here for several reasons:

1. Previous thread become too big to follow it
2. While this topic is related to original discussion it's actually important enough to be separated
3. 21 days passed from the moment I posted it, 16 from @erights promise to review it - and NO response so far (even though he was active in https://github.com/rdking/proposal-class-members/issues/10#issuecomment-445988975)
4. According to this: https://github.com/tc39/proposal-class-fields/issues/175#issuecomment-441698604, https://github.com/tc39/proposal-class-fields/issues/149#issuecomment-431527298 and September meeting it seems to be a major concern against Symbol.private
   • Unfortunately, there are no notes from latest meeting where it was discussed
   • there is also concern about lack of ergonomic syntax, but I already proposed few solutions (e.g. #149, #134 and https://github.com/tc39/proposal-class-fields/issues/149#issuecomment-442242427), and I'm not the only one who did it (e.g. https://github.com/rdking/proposal-class-members/issues/10), so IMO it could be solved if Symbol.private will reach stage 1, so we'll be able to work on it or create follow-on proposals

@ljharb, @erights, @littledan, @zenparsing could you, please answer to this:

1. Is Membrane pattern a major concern?
2. Did I address Membrane+Symbol.private issues with following solution?
3. Is lack of ergonomic syntax is also a major concern?
4. Should I summarize syntax solutions for Symbol.private (including my yet unpublished ideas)?

Copied from https://github.com/tc39/proposal-class-fields/issues/149#issuecomment-441057730

---

@erights, as I said before there is a solution that makes your test reachable using Symbol.private, here's a sample:

function isPrimitive(obj) {
    return obj === undefined
        || obj === null
        || typeof obj === 'boolean'
        || typeof obj === 'number'
        || typeof obj === 'string'
        || typeof obj === 'symbol'; // for simplicity let's treat symbols as primitives
}

function createWrapFn(originalsToProxies, proxiesToOriginals, unwrapFn) {
    // `privateHandlers` are special objects dedicated to keep invariants built
    // on top of exposing private symbols via public API
    // we also need one-to-one relation between `privateHandler` and `original`
    const privateHandlersOriginals = new WeakMap();
    // we're keep track of created handlers, so we'll be able to adjust them with
    // newly exposed private symbols
    const allHandlers = new Set();

    // just simple helper that creates getter/setter pair for specific
    // private symbol and object that gets through membrane
    function handlePrivate(handler, privateSymbol) {
        const original = privateHandlersOriginals.get(handler);
        Object.defineProperty(handler, privateSymbol, {
            get() {
                return wrap(original[privateSymbol]);
            },
            set(v) {
                original[privateSymbol] = unwrapFn(v);
            }
        })
    }

    function wrap(original) {
        // we don't need to wrap any primitive values
        if (isPrimitive(original)) return original;
        // we also don't need to wrap already wrapped values
        if (originalsToProxies.has(original)) return originalsToProxies.get(original);
        const privateHandler = {};
        privateHandlersOriginals.set(privateHandler, original);
        allHandlers.add(privateHandler);

        //       note that we don't use `original` here as proxy target
        //                      ↓↓↓↓↓↓↓↓↓↓↓↓↓↓
        const proxy = new Proxy(privateHandler, {
            apply(target, thisArg, argArray) {
                thisArg = unwrapFn(thisArg);
                for (let i = 0; i < argArray; i++) {
                    if (!isPrimitive(argArray[i])) {
                        argArray[i] = unwrapFn(argArray[i]);
                    }
                }

                //          but we use `original` here instead of `target`
                //                           ↓↓↓↓↓↓↓↓
                const retval = Reflect.apply(original, thisArg, argArray);

                // in case when private symbols is exposed via some part of public API
                // we have to add such symbol to all possible targets where it could appear
                if (typeof retval === 'symbol' && retval.private) {
                    allHandlers.forEach(handler => handlePrivate(handler, retval));
                }

                return wrap(retval);
            },
            get(target, p, receiver) {
                receiver = unwrapFn(receiver);
                //       but we use `original` here instead of `target`
                //                         ↓↓↓↓↓↓↓↓
                const retval = Reflect.get(original, p, receiver);

                // in case when private symbols is exposed via some part of public API
                // we have to add such symbol to all possible targets where it could appear
                if (typeof retval === 'symbol' && retval.private) {
                    allHandlers.forEach(handler => handlePrivate(handler, retval));
                }

                return wrap(retval);
            },
            // following methods also should be implemented,
            // but it they are skipped for simplicity
            getPrototypeOf(target) { },
            setPrototypeOf(target, v) { },
            isExtensible(target) { },
            preventExtensions(target) { },
            getOwnPropertyDescriptor(target, p) { },
            has(target, p) { },
            set(target, p, value, receiver) { },
            deleteProperty(target, p) { },
            defineProperty(target, p, attributes) { },
            enumerate(target) { },
            ownKeys(target) { },
            construct(target, argArray, newTarget) { },
        });

        originalsToProxies.set(original, proxy);
        proxiesToOriginals.set(proxy, original);

        return proxy;
    }

    return wrap;
}

function membrane(obj) {
    const originalProxies = new WeakMap();
    const originalTargets = new WeakMap();
    const outerProxies = new WeakMap();

    const wrap = createWrapFn(originalProxies, originalTargets, unwrap);
    const wrapOuter = createWrapFn(outerProxies, originalProxies, wrap)

    function unwrap(proxy) {
        return originalTargets.has(proxy)
            ? originalTargets.get(proxy)
            : wrapOuter(proxy);
    }

    return wrap(obj);
}

const privateSymbol = Symbol.private();
const Left = {
    base: {
        [privateSymbol]: ''
    },
    value: '',
    field: privateSymbol,
};
const Right = membrane(Left);

const { base: bT, field: fT, value: vT } = Left;
const { base: bP, field: fP, value: vP } = Right;

// # set on left side of membrane
// ## set using left side field name
bT[fT] = vT;
assert(bP[fP] === vP);

bT[fT] = vP;
assert(bP[fP] === vT);

// ## set using right side field name
bT[fP] = vT;
assert(bP[fT] === vP);

bT[fP] = vP;
assert(bP[fT] === vT);

// # set on right side of membrane
// ## set using left side field name
bP[fT] = vT;
assert(bT[fP] === vP);

bP[fT] = vP;
assert(bT[fP] === vT);

// ## set using right side field name
bP[fP] = vT;
assert(bT[fT] === vP);

bP[fP] = vP;
assert(bT[fT] === vT);

Copied from https://github.com/tc39/proposal-class-fields/issues/149#issuecomment-441075202

---

I skipped fP->fT and fT->fP transformations in previous example for simplicity. But I'll mention such transformation here, since they are really easy handled when private symbols crosses boundary using public API of membraned object

set on left side of membrane

set using left side field name

bT[fT] = vT;

goes as usual, since happens on one (left) side of membrane

assert(bP[fP] === vP);

• since fP is private symbol, get called on proxy target
• which is special privateHandler object
• call privateHandler[fP] getter:
  • unwrap fP to fT
  • get bT[fT] which equals vT
  • wrap vT to vP

• assertion is TRUE

  ---

bT[fT] = vP;

• unwrap vP to vT
• set vT to bt[fT] as usual

  assert(bP[fP] === vT);

• proxy skipped to target (privateHandler)
• call privateHandler[fP] getter:
  • unwrap fP to fT
  • get bT[fT] which equals vT
  • wrap vT to vP
• wrap vT to vP
• assertion is TRUE

set using right side field name

bT[fP] = vT;

• unwrap fP to fT
• set vT to bt[fT] as usual

  assert(bP[fT] === vP);

• wrap fT to fP
• proxy skipped to target (privateHandler)
• call privateHandler[fP] getter:
  • unwrap fP to fT
  • get bT[fT] which equals vT
  • wrap vT to vP

• assertion is TRUE

  ---

bT[fP] = vP;

• unwrap fP to fT
• unwrap vP to vT
• set vT to bt[fT] as usual

  assert(bP[fT] === vT);

• wrap fT to fP
• proxy skipped to target (privateHandler)
• call privateHandler[fP] getter:
  • unwrap fP to fT
  • get bT[fT] which equals vT
  • wrap vT to vP
• wrap vT to vP
• assertion is TRUE

set on right side of membrane

set using left side field name

bP[fT] = vT;

• wrap fT to fP
• wrap vT to vP
• set vP to bP[fP]
• proxy skipped to target (privateHandler)
• call privateHandler[fP] setter:
  • unwrap fP to fT
  • unwrap vP to vT
  • set vT to bt[fT] as usual

    assert(bT[fP] === vP);

• unwrap fP to fT
• get bT[fT] which equals vT
• unwrap vP to vT

• assertion is TRUE

  ---

bP[fT] = vP;

• wrap fT to fP
• set vP to bP[fP]
• proxy skipped to target (privateHandler)
• call privateHandler[fP] setter:
  • unwrap fP to fT
  • unwrap vP to vT
  • set vT to bt[fT] as usual

    assert(bT[fP] === vT);

• unwrap fP to fT
• get bT[fT] which equals vT
• assertion is TRUE

set using right side field name

bP[fP] = vT;

• wrap vT to vP
• set vP to bP[fP]
• proxy skipped to target (privateHandler)
• call privateHandler[fP] setter:
  • unwrap fP to fT
  • unwrap vP to vT
  • set vT to bt[fT] as usual

    assert(bT[fT] === vP);

• get bT[fT] which equals vT
• unwrap vP to vT

• assertion is TRUE

  ---

bP[fP] = vP;

• set vP to bP[fP]
• proxy skipped to target (privateHandler)
• call privateHandler[fP] setter:
  • unwrap fP to fT
  • unwrap vP to vT
  • set vT to bt[fT] as usual

    assert(bT[fT] === vT);

• get bT[fT] which equals vT
• assertion is TRUE

[jridgewell]

and just real target membranes. What is that?

new Proxy(target, {}), where target is from the other side of the membrane. Vs, new Proxy(shadow, {}), where shadow is on this side of the membrane and mirrors target on the other side.

This was to prove that a shadow target design weren't necessary for the membrane implementation (though they may be necessary to accomplish things like distortions). Just that private symbols doesn't force this decision.

Leaving aside decorators and looking only at the current stage 3 proposal, is there anything in your proposal that is incompatible with it?

It's largely compatible with the current proposal, to make it more palatable. The changes are:

• Private symbols would be prototype based like public symbols
• They can be installed on extensible foreign objects, like public symbols
• Proxies doen't trap (this is ignoring whitelist, since it's deferred)

Further, the design can be used to accomplish "branding" as in the current proposal, allowing us to still support spec internal slots:

class Branded {
  #brand = this;
  #priv = someData;

  static #brandCheck(obj) {
    if (obj.#brand !== obj) {
      throw new TypeError('non instance');
    }
  }

  methodThatChecksBrand() {
    // Assert Branded's constructor ran on `this`,
    // giving the same semantics as current class fields proposal.
    Branded.#checkBrand(this);

    // safe to use #priv
    this.#priv.something();
  }

Btw, we need to ask the same question for PNLSOWNSF.

Is PNLSOWNSF mean a reified PrivateName without proposing instance.#field syntax?

[ljharb]

How would extensibility factor in to private symbols? A nonextensible class instance can still have its private fields modified, and if a superclass made the instance non-extensible i assume the subclass’s constructor could still install private fields on it. Extensibility relates to public properties only, in my understanding.

[rdking]

@ljharb Would that still be true for private fields if they were actual properties of the object? The only difference between public and private fields when using private symbols is that the name of the property is undiscoverable. Since all properties would be in the same bag, it would be strange if a flag that affects the whole bag somehow missed private symbol-keyed properties.

[erights]

new Proxy(target, {}), where target is from the other side of the membrane. Vs, new Proxy(shadow, {}), where shadow is on this side of the membrane and mirrors target on the other side.

This was to prove that a shadow target design weren't necessary for the membrane implementation (though they may be necessary to accomplish things like distortions). Just that private symbols doesn't force this decision.

I am glad for this point. But independent of your point, to be pedantic, I don't think it is possible to build a membrane using real targets as shadows. If someone does see a way to do that, even if impractical, I would be interested and would probably learn something. Thanks.

[erights]

They can be installed on extensible foreign objects, like public symbols

If "They" means private symbols, how can this be a compatibility issue before private symbols are introduced?

[erights]

OMG, I just realized that this still suffers from the problem from the last breakout group: You're still vulnerable to confused deputy, right? You're not separating initialization from assignment:

class Foo {
  #state;
  ...
  setState(newState) {
    this.#state = newState;
  }
}

const f = new Foo();
const notAFoo = {};
f.setState.call(notAFoo);

Does this result in #state being installed on notAFoo? This is fatal.

I think probably all the private symbol variants have this problem. The PNLSOWNSF can avoid it. Without thinking about it, I was assuming they would, but this takes extra mechanism --- part of why PNLSOWNSF isn't a concrete proposal yet. In any case, having turned my attention to PNLSOWNSF I missed till now that the private symbol approaches still had this fatal problem.

[erights]

The extensibility issue hangs critically on the issue of where the mutable state resides. If I freeze and object with no hidden state, I make it immutable. If I use it as a key in a WeakMap, I can have a mutable mapping from the object to a value "named" by that WeakMap, because the mutability is in the WeakMap.

Private symbols are primitives, and so stateless. If an immutable object can be extended with a new "name" to value mapping, where is the mutable state that got mutated to record this association? If it is further a mutable mapping to value, where does that mutable state reside?

[jridgewell]

How would extensibility factor in to private symbols?

Because an isExtensible check happens when creating new properties. I'm not (currently) proposing we change that for private symbols.

A nonextensible class instance can still have its private fields modified

Talking about frozen instances, yes. Because Object.freeze can't find the property to set it to non-writable.

and if a superclass made the instance non-extensible i assume the subclass’s constructor could still install private fields on it.

Not under the current design. If this is an important goal, we can special case private symbols to skip the isExtensible check. But I like that private symbols behave very similarly to public symbols, and if I can't install a public field I wouldn't expect to be able to install a private field.

I don't think it is possible to build a membrane using real targets as shadows.

My implementation does, though, it's not a full implementation. It just does get, set, and apply, since they were what's needed for the membrane invariants. If you can think of any cases that could be accomplished with shadow targets and not real targets, I'd be happy to add more to the test suite.

You're still vulnerable to confused deputy, right? You're not separating initialization from assignment:

Correct, by design. But from my last comment, you can accomplish branded checks to overcome this using the private symbol syntax.

The extensibility issue hangs critically on the issue of where the mutable state resides... Private symbols are primitives, and so stateless. If an immutable object can be extended with a new "name" to value mapping, where is the mutable state that got mutated to record this association? If it is further a mutable mapping to value, where does that mutable state reside?

This is exactly why private symbols cannot be added to non-extensible objects. I'm of the opinion that instance.#field = value looks like instance.field = value, and so should behave similarly. That would imply instance holds the state.

Were we to use the abstract ref's instacne::map/instance::map = value syntax instead, I wouldn't be pushing for private symbols at all.

[erights]

Is PNLSOWNSF mean a reified PrivateName without proposing instance.#field syntax?

Time permitting, I will try to sketch a concrete PNLSOWNSF straw proposal tomorrow. (Rather than break expectations again, I'll note now that I might not find the time tomorrow or for awhile.)

@rdking @zenparsing , you each seemed in tune with PNLSOWNSF . I would be interested to see what concrete sketches you come up with. And it would give this conversation some new non-private-symbol points in the design space. Thanks.

[erights]

My implementation does, though, it's not a full implementation. It just does get, set, and apply, since they were what's needed for the membrane invariants. If you can think of any cases that could be accomplished with shadow targets and not real targets, I'd be happy to add more to the test suite.

I'm curious if you can do the following without separate shadows, and what is printed. Thanks.

const blueBob = {};
const blueAlice = Object.freeze({bob: blueBob});
const yellowAlice = membrane(blueAlice, whatever...);
const desc = Reflect.g22r(yellowAlice.bob);
console.log(desc.value === blueBob, desc.configurable, desc.writable);

[erights]

Were we to use the abstract ref's instacne:map/instance:map = value syntax instead, I wouldn't be pushing for private symbols at all.

Do you mean @zenparsing 's double colon syntax :: ?

[erights]

Single colon would almost certainly create visual confusion with the existing uses of single colon in both JS and TypeScript.

[ljharb]

@jridgewell i mean, under the current design of class fields, a nonextensible instance from a superclass can still receive private fields, no? If not, I’ll file an issue about that, because it violates weakmap-like semantics; if so, i don’t see how private symbols can ever be affected by extensibility.

[erights]

You're still vulnerable to confused deputy, right? You're not separating initialization from assignment:

Correct, by design. But from my last comment, you can accomplish branded checks to overcome this using the private symbol syntax.

Please rewrite

class Foo {
  #state;
  ...
  setState(newState) {
    this.#state = newState;
  }
}

const f = new Foo();
const notAFoo = {};
f.setState.call(notAFoo);

so that it would be safe under your proposal. Thanks.

Assuming the rewrite is as ugly as I expect:

My position on this is unchanged from the breakout session at the last meeting. The easy way should be the safe way. Or at least, the safe way should not be so much harder that no one habituates to using it in normal code. If the safe way is less readable, then that alone makes it less safe. Many others there were also clear that integrity-by-default is a requirement. I cannot see all of us changing our mind on that.

[erights]

Were we to use the abstract ref's instacne:map/instance:map = value syntax instead, I wouldn't be pushing for private symbols at all.

@jridgewell , Assuming you mean @zenparsing's ::, this raises the following question:

If we took exactly the current stage 3 proposal, changed none of its semantics. Zero. But changed its surface syntax so that every .# were spelled :: instead, and the remaining # for declarations were also spelled ::, would that address your concerns?

[rdking]

@erights It's not difficult at all.

class Foo {
  #brand;
  #state;
  ...
  setState(newState) {
    this.#brand; //It either exists or it faults.
    this.#state = newState;
  }
}

const f = new Foo();
const notAFoo = {};
f.setState.call(notAFoo); //Faults because notAFoo doesn't have #brand.

[erights]

@rdking That's exactly what I expected. No one will write that in normal code. It is a non-starter.

[rdking]

Funny considering I've done that before when I needed to be sure of the context object. The only difference is that I used a Symbol.

[rdking]

@erights Take into account that where brand checking is currently needed, exactly that kind of thing is being done. To blithely state that "no one will write that" is a very grand assumption, and demonstrably false.

[jridgewell]

I'm curious if you can do the following without separate shadows, and what is printed. Thanks.

Wow, that throws. I wasn't aware of the proxy invariant:

The value reported for a property must be the same as the value of the corresponding target object property if the target object property is a non-writable, non-configurable own data property. - https://tc39.github.io/ecma262/#sec-proxy-object-internal-methods-and-internal-slots-get-p-receiver

That seems a strange invariant. But that just means that shadow targets are definitely necessary.

Do you mean @zenparsing 's double colon syntax :: ?

Correct, updated my comment.

Please rewrite so that it would be safe under your proposal.

class Foo {
  #state;
  #brand = this;

  setState(newState) {
    if (this.#brand !== this) throw new TypeError();
    this.#state = newState;
  }
}

const f = new Foo();
const notAFoo = {};
f.setState.call(notAFoo);

The easy way should be the safe way. Or at least, the safe way should not be so much harder that no one habituates to using it in normal code. If the safe way is less readable, then that alone makes it less safe. Many others there were also clear that integrity-by-default is a requirement.

This is definitely a trade-off. My way is less secure by default, but can be just as secure with boilerplate (a point I'll make during the discussion).

I don't think, however, that users will depend much on brand semantics. Large end projects will will have a type system that eliminates the need for ever having a runtime brand check (and accomplishes sooo much more).

Smaller end projects won't really need a brand check, because they control the code. They'd be attacking themselves.

It's only library authors who cannot depend on a type system, and who may wish to secure their code against end users. I don't think the boilerplate is too terrible in this circumstance. I could even see a build step that adds manual checks automatically.

And note, the likely place for this manual check is at the beginning of the method. I think this is better than at the first this.#field property use. It makes it clear, and prevents me from running any code between the start of the method and the first branded property use (like access public fields, or anything else). And branding also doesn't do anything for the public shape of the object, only the private shape. I like the idea of separating out these concerns, so some later proposal can achieve a better branding outcome.

I cannot see all of us changing our mind on that.

I may not convince everyone. But the discussion is still worth having now that we can dismiss membranes as a technical limitation.

If we took exactly the current stage 3 proposal, changed none of its semantics. Zero. But changed its surface syntax so that every .# were spelled :: instead, and the remaining # for declarations were also spelled ::, would that address your concerns?

Yes. It would look very different than instance.field access, so I would not assume the same access semantics apply (I even like the design of using weakmap-mutable state, it's a useful idea!). But I really think that confusing devs with two similar-looking-but-different access semantics is a bad idea.

---

a nonextensible instance from a superclass can still receive private fields, no?

I'm actually not certain.

If not, I’ll file an issue about that, because it violates weakmap-like semantics; if so, i don’t see how private symbols can ever be affected by extensibility.

My current design has chosen not to skip the isExtensible semantic. This is a noted difference in the two. 😉

[rdking]

@jridgewell

That seems a strange invariant. But that just means that shadow targets are definitely necessary.

The main reason shadow targets are necessary is that Proxy doesn't self-check. The invariants themselves all makes sense, but all of the invariants test against the Proxy's first parameter instead of against the handler. This prevents the Proxy from presenting a self-consistent view that disagrees with critical features the original object. But I guess that's a problem for later.

[rdking]

@erights

If we took exactly the current stage 3 proposal, changed none of its semantics. Zero. But changed its surface syntax so that every .# were spelled :: instead, and the remaining # for declarations were also spelled ::, would that address your concerns?

Ignoring the syntax appearance improvement, that would make it clear that what is being accessed is something very different from a normal(public) property. I would drop #202 in trade for that change.

[ljharb]

@jridgewell if that’s a difference, then your design isn’t a reified form of this proposal, so I’m not sure how it could achieve consensus :-/

[jridgewell]

if that’s a difference, then your design isn’t a reified form of this proposal, so I’m not sure how it could achieve consensus :-/

I don't understand. My design also traverses the prototype, which class-fields doesn't. I have intentionally made different decisions that make my proposal behave more like regular properties, and shown that they can still achieve the membrane invariants. That was my goal.

[ljharb]

Gotcha. I misunderstood then; i thought you were proposing that private fields would become reifiable as private symbols.

[rdking]

@erights

@rdking @zenparsing , you each seemed in tune with PNLSOWNSF . I would be interested to see what concrete sketches you come up with

I don't actually like PNLSOWNSF. It's a ok concept with interesting effects, but it seems wasteful to me. It reverses the object/key relationship causing the "key" to be the object maintaining the state information. Essentially, each key is a WeakMap, the object is the key to that WeakMap, and a proliferation of such keys incurs a heavy cost.

It's much simpler, and already built into the language to let the object maintain its own state, and keep keys as simple as possible. Adding the ability to hide a key from Reflection and Proxy is far simpler, faster, and less memory intensive than PLNSOWNSF.

---

As a side note, given that any private field using subclass under class-fields can fall victim to the confused-deputy problem, I don't understand why it's such a sticking point. Can someone explain the reason for me?

[jridgewell]

i thought you were proposing that private fields would become reifiable as private symbols.

Oh, nope! The current class-fields would be really odd if we tried reifying to private symbols. We can make other changes that make it behave more like prototype properties while keeping the map interface. But I definitely think the map interface is the correct reification for the current proposal.

[erights]

But the discussion is still worth having now that we can dismiss membranes as a technical limitation.

Are you still tunneling? If g22r of a private symbol tunnels through to the shadow, how does the membrane first update the state of the shadow to reflect the current state of the real target before the tunneling gets the descriptor from the shadow? Likewise, if defineProperty tunnels, how does the membrane update the real target afterwards, to reflect the updated state of the shadow?

[erights]

In short, "dismiss" is too strong a word. You've made great progress on membranes, establishing the plausibility of solving all hard constraints. Kudos! But these solutions come at a considerable cost in extra complexity, in an area that is already too hard to reason about. We should not yet be anywhere close to confident that we've really figured it out. It took us years to become reasonably confident about the existing proxy design.

"dismiss" raises hackles. I agree that membranes are no longer the main objection.

[jridgewell]

If g22r of a private symbol tunnels through to the shadow, how does the membrane first update the state of the shadow to reflect the current state of the real target before the tunneling gets the descriptor from the shadow? Likewise, if defineProperty tunnels, how does the membrane update the real target afterwards, to reflect the updated state of the shadow?

That's the beauty of the design :smile:.

The two cases you mention here are only interesting when a reified private symbol crosses the membrane. Ie, I'm on the Left using a right-created private symbol, or on the Right using a left-created one. With syntax-based private symbols only, only a closure could pass through the membrane, and that should already be handled by wrapping the closure as if it were any other closure to wrap/unwrap its parameters. This is part of the reason I think we should only ship syntax first.

In both cases, because it crossed the membrane, the membrane can take the necessary actions to update everything. https://github.com/jridgewell/membrane-test-for-symbol-private/blob/f2c526585c82db577e92ab55d08a9b87db883a2d/index.js#L118-L121

One step is to add the private symbol to the known whitelist. When any access happens on proxy-wrapped objects in the future, the trap will handle this. https://github.com/jridgewell/membrane-test-for-symbol-private/blob/f2c526585c82db577e92ab55d08a9b87db883a2d/index.js#L78

But this alone isn't enough. I could have Left (or wrapped-right) values transparently installed on wrapped-right target, with the setting done before the membrane knew about the private symbol (ie, not in the whitelist). This is the bP[fT] = vT and bP[fT] = vP cases. Here, once the private symbol crosses the membrane, I also need up sync the values stored on the shadow targets to be appropriate wrapped (or unwrapped) on the real target. In the case of a left-created private symbol being pulled to the right side, I need to look over all right-created objects that passed through the membrane to the Left side to see if they contain that symbol. https://github.com/jridgewell/membrane-test-for-symbol-private/blob/f2c526585c82db577e92ab55d08a9b87db883a2d/index.js#L80-L86

This does introduce a need for strongly holding onto the original objects inside the membrane. This poses a GC issue. But, WeakRefs make that easy.

[erights]

I wrote

Time permitting, I will try to sketch a concrete PNLSOWNSF straw proposal tomorrow.

See https://github.com/erights/PLNSOWNSF . Still rough.

Whew! Sleepy now.

[erights]

attn @wycats

[zenparsing]

Thanks @erights, I will review and comment on that repo in time.

For now, let me say a couple of things:

• I agree that we probably want something WeakMap-like rather than WeakMap itself (mostly for GC/performance reasons).
• The :: syntax is close but isn't quite what I have in mind; more below.
• We don't need to use fields or invent the parallel namespace. Normal variables, with the right syntax support, are quite enough.

A preview:

import { hiddenState } from 'hidden-state';

const [getState, initState, isInstance] = hiddenState('Point'); 

class Point {

  constructor(x, y) {
    this->initState({ x, y });
  }

  toString() {
    let { x, y } = this->getState();
    return `<${x}:${y}>`;
  }

  add(p) {
    let { x, y } = this->getState();
    let { x: x1, y: y1 } = p->getState();
    return new Point(x + x1, y + y1);
  }

  static isPoint(x) {
    return x->isInstance();
  }

}

Notes:

• See https://github.com/zenparsing/hidden-state for the defintion of the "hidden-state" module. It is currently backed by a WeakMap and throws TypeErrors like you want.
• The -> syntax is quite simple: it merely calls the RHS with the LHS as a first argument.

Cheers!

[littledan]

I think it's useful to have analogous syntax between public and private fields (as well as public and private methods): This way, you can refactor between public and private just by changing the name of the field (or method), and leaving the rest of the code the same. You don't need to move the field initializer into or out of the constructor (or reconsider what the this value is in methods). Many other object-oriented languages work towards this analogy as well; let's follow their experience.

I'm skeptical of using ->. The experience from C++ shows that it's annoying to have to juggle two different infix operators for accessing elements of an object, even when they do two very different things.

[zenparsing]

I'm skeptical of using ->. The experience from C++ shows that it's annoying to have to juggle two different infix operators for accessing element

In the example above, -> is not used for accessing elements. It is used to call a function, providing the LHS as the first argument.

I think it's useful to have analogous syntax between public and private fields

Let me share my thoughts here:

JavaScript does not have "fields" of course; it has properties.

The whole premise of private fields from the very beginning was that we could introduce a property-like syntax for accessing completely private state, ala internal slots. The trouble is, the more property-like you make it, the less private it is, and vice-versa.

Private fields prioritizes privacy over property-ness, and that creates all of the "less that ideal" interactions with other language features. (All of the issues that I presented in NYC and that have been discussed at length here.)

Private symbols prioritizes property-ness. They integrate much better with other language features, but at the expense of privacy (in the sense that you can observe prototype chain walks and the lack of "brand checking").

As the original author of the private fields proposal, I've spent quite a bit of time in this area, and I have come to the conclusion that any solution which attempts to create property-like syntax for completely private state is doomed to "fall between the stools". Like the computer in War Games, I understand now that the only way to win is not to play.

Plus, the example I posted above shows that WeakMap (or PrivateName) backed private state, with the right abstractions around it and syntactic support, can be very ergonomic and readable. This isn't a zero-sum game.

Thanks for listening!

[littledan]

(Oh, I think you're using Data and State as the same thing in your example; you might want to fix that for clarity.)

[zenparsing]

Thanks @littledan, fixed : )

[caridy]

yeah, noticed that as well in the example.

@zenparsing one thing to keep in mind is branding. In your example above, it is mostly a weakmap-like where branding must be intentional. Lets say you have a move method on that class:

  move(x, y) {
    this->setState({ x, y });
  }

How can I prevent someone calling originalPoint.move.call({}), which as a result, will add the new object to the weakmap? this is something that we discussed in the past, and seem to be very important. In the case of the private fields proposal, we are getting that, but was one of the major problems when we tried to generalize those fields. Do you have any idea how that can be done with this new syntax? @erights are you ok without having such capability?

[littledan]

Note, if we're talking about "branding", please note the clarification in the FAQ about exactly what guarantees the current proposal provides.

I like to think of the practical benefit here being avoiding errors in programming that would otherwise be likely, whereas the most important strong guarantee from my perspective is about privacy being maintained (which the hidden state proposal preserves, but which I believe @zenparsing argued against elsewhere). You can build a stronger branding guarantee for subclasses by freezing the subclass constructor, though.

[zenparsing]

Hi @caridy,

In the example above, a move method should be implemented as follows:

  move(x, y) {
    let state = this->getState();
    state.x = x;
    state.y = y;
  }

It would probably be better to rename setState to initState in order to clarify its intent.

[jridgewell]

I think it's useful to have analogous syntax between public and private fields (as well as public and private methods): This way, you can refactor between public and private just by changing the name of the field (or method), and leaving the rest of the code the same.

Except for any of the footguns. Private fields aren't properties like public fields.

Private symbols prioritizes property-ness. They integrate much better with other language features, but at the expense of privacy (in the sense that you can observe prototype chain walks and the lack of "brand checking").

At the expense of privacy (under those two criteria), but not at the expense of encapsulation. Only sourcecode that has the private symbol (either syntatic or reified) in lexical scope can access/modify the property. I think that's still pretty good.

[rdking]

@erights

But these solutions come at a considerable cost in extra complexity, in an area that is already too hard to reason about.

If Membranes are "already too hard to reason about", then why not just offer Membrane as built-in? In my mind it defies reason that Proxy was created with the sole intention of allowing Membrane, but Membrane itself, being a complicated concept not easily implemented if at all by most developers, is not provided when it was the goal.

[rdking]

@littledan

I'm skeptical of using ->. The experience from C++ shows that it's annoying to have to juggle two different infix operators for accessing elements of an object, even when they do two very different things.

At the same time, we have obj.name and will have obj.#name. From the perspective of some developers, it already looks like we're doing 2 operators for property access.

[littledan]

Except for any of the footguns. Private fields aren't properties like public fields.

"Footgun" is pretty strong language to use; we've really worked hard to eliminate as many unexpected cases as possible.

For the most part, private fields and methods provide a strict subset of the semantics of public fields and methods, with exceptions thrown for the cases where you can't do something with private. That doesn't explain 100% of everything, but when people need the advanced explanation for complex cases, the WeakMap story is there to clarify. (There are going to be multiple levels of detail that people go into in understanding language features, whether we hope for it or not.)

Imagine if we applied this logic for lexically scoped variables. They don't hoist--this is a footgun--we should use a different syntax for them, that's not just a var replacement keyword, to teach people a lesson!

[erights]

If Membranes are "already too hard to reason about", then why not just offer Membrane as built-in? In my mind it defies reason that Proxy was created with the sole intention of allowing Membrane, but Membrane itself, being a complicated concept not easily implemented if at all by most developers, is not provided when it was the goal.

Right now, membranes are a pattern. Distortions are expressed by varying the pattern, so there is no one clear thing to build in. @ajvincent's ES-Membrane Library shows the way forward: A membrane-creating system as a resuable abstraction mechanism, to be parameterized by distortions. At my invitation, Alex presented on this at tc39. I do hope that, eventually, we will have a good understanding of an algebra of distortion composition, leading to convergence on a good API for parameterizing a reusable membrane library. However, this might take years of research. Nevertheless, I am hopeful that, eventually, we will have consensus on what mechanisms should be standardized and built in to provide more direct support for membranes.

A comparison:

We have RegExp as a builtin pattern matching abstraction mechanism, for generating pattern matchers from a grammar describing what to recognize. However, we still build parsers by hand. There are so many different forms of parsers and parser generators that we can only say that the general idea of bnf-based parsing is a pattern, not currently a candidate for a builtin. Why make everyone write their own parser or parser generator from scratch?

Were a particular bnf-like grammar description language and resulting parser behavior to rise above the others, achieving a consensus shared sense of good-enough, we might indeed eventually consider standardizing a parser generator --- an abstraction mechanism to be parameterized by grammars and semantic actions --- and building it in, so everyone can stop rolling their own.

This is not merely a rhetorical example. Template literal tags are our most powerful tool for avoiding injection attacks (attn @jugglinmike ). However, they are underused because of insufficient support for creating tags for interesting languages. I hope that eventually https://github.com/erights/quasiParserGenerator (see also https://github.com/michaelfig/jessica ) may lead to such a standardized builtin mechanism. As with membranes, a standardized template-literal-tag generator is also probably years away. For now, it remains a pattern.

[littledan]

I'm also not convinced that a more differentiated token like -> or :: would give people stronger intuition for the different semantics than .# would. People would still draw the connection, as you're still getting a named thing from the object, somehow or other. They would just have two problems: potentially mixing up the syntax (because of the loss of the "# is part of the name" property) and also remembering the semantics, vs just the one problem of remembering the semantics.

For example, if you program C++, and ever typed code with . where you meant ->, or vice versa (where the semantics are very different--de-referencing a pointer is sort of a big deal, and pointers are one of the first things you have to learn to be able to program in C++), you've experienced this.

[erights]

The -> syntax is quite simple: it merely calls the RHS with the LHS as a first argument.

How does this relate to any of the proposed pipeline syntaxes? IMO, I am skeptical that any of the pipeline syntaxes pull their weight. But if the introduction of a basic simple pipeline operator syntax can also subsume our need for .#, then it would likely be a net win.

[jridgewell]

"Footgun" is pretty strong language to use; we've really worked hard to eliminate as many unexpected cases as possible.

It's absolutely a footgun. A known pattern for static public fields will not work when made private. This also applies if we were to ever extend this to object literals, because static inheritance and object inheritance are the same thing.

Imagine if we applied this logic for lexically scoped variables. They don't hoist--this is a footgun--we should use a different syntax for them, that's not just a var replacement keyword, to teach people a lesson! (because of the loss of the "# is part of the name" property

These two statements don't jive. We have INSTANCE . PROPERTY_NAME for both public and private (because the # is part of the name). They'll behave very differently. That is not a good idea.

Private symbols can provide a solution that gives us the least difference between the two (the goal is encapsulation, so there must be some difference).

How does this relate to any of the proposed pipeline syntaxes? ... But if the introduction of a basic simple pipeline operator syntax can also subsume our need for .#, then it would likely be a net win.

This -> pipeline is using Elixir style semantics. There's a tracking issue for it at https://github.com/tc39/proposal-pipeline-operator/issues/143.

[Igmat]

First of all, I'm glad that we have meaningfull discussion on this topic at last.

You have to know what these are up front when you create the membrane. Or did I misunderstand?

@erights, yes you misunderstood - we don't have to know all possible Symbol.private's before creating Membrane, we'll use only once that are exposed by one or another side. I think that @jridgewell has described it full enough in his https://github.com/tc39/proposal-class-fields/issues/183#issuecomment-451713360. Do you need more explenations for this question?

Also, while I fully agreed with @jridgewell that we have to stop discussing my implementation (since it doesn't work in frozen context, because I wasn't aware of such requirement), I want to point out that such implementation could still be usable for some circumstances, before whitelisting Proxy is released. Does it make any sense for you?

I agree that membranes are no longer the main objection.

Is it possible to use this quote of yours as an argument in favor of Symbol.private proposal?

Please rewrite

class Foo {
  #state;
  ...
  setState(newState) {
    this.#state = newState;
  }
}

const f = new Foo();
const notAFoo = {};
f.setState.call(notAFoo);

so that it would be safe under your proposal. Thanks.

Assuming the rewrite is as ugly as I expect:

My position on this is unchanged from the breakout session at the last meeting. The easy way should be the safe way. Or at least, the safe way should not be so much harder that no one habituates to using it in normal code. If the safe way is less readable, then that alone makes it less safe. Many others there were also clear that integrity-by-default is a requirement. I cannot see all of us changing our mind on that.

Your position stands on the assumption that #state ALWAYS intented by developer as private part of implementation that MUST NOT be installed to any other object. This is only an assumption, and it definitely not true for all cases. Simple sample of code, where developer has real intent to install Symbol.private to another object.

const cacheSym = Symbol.private();

class A {
    // or something like this
    #cacheSym;
    requestSomething(someObjectThatDefinesRequrest) {
        if (someObjectThatDefinesRequrest[cacheSym]) return someObjectThatDefinesRequrest[cacheSym];
        // some complex logic for requesting and calculating request result
        someObjectThatDefinesRequrest[cacheSym] = result;
        return result;
    }
}

Actually, this pattern is very usefull by itself, and could be used as replacement for a lot of WeakMap usages, while being also Proxy safe and requiring less memory consumption and less complicated work of GC. Do you agree with this statement? And did @jridgewell adressed requirement for brand-checking when need in his https://github.com/tc39/proposal-class-fields/issues/183#issuecomment-451719147?

@littledan, answering to this part of your comment:

I like to think of the practical benefit here being avoiding errors in programming that would otherwise be likely

Eliminating some typo errors (and I think you want argue that linters and type checkers do it MUCH better) by adding IMPLICIT brand-check causes losing of a huge variety of usefull patterns (e.g. one shown above). Do you think that it's reasonable trade-off?

P.S.

I just realized that Symbol.private could be used for implementing Membrane pattern instead of WeakMap's and it probably could also eliminate some issues and make things easier to reason about, but I have to investigate it a little bit more - I'll share my results in this field a little bit later.

[littledan]

It's absolutely a footgun. A known pattern for static public fields will not work when made private. This also applies if we were to ever extend this to object literals, because static inheritance and object inheritance are the same thing.

We spent several (8?) months going through this issue that you raised. I gave you a lot of space for that investigation, including the unusual step of retracting the proposal to Stage 2 for the long investigation. Eventually, we (including you) agreed that the original proposal should go back to Stage 3. I'm surprised that, after all that, still saying here that it's a problem.

This case matches the pattern I explained above: It's like public when things are allowed (for the most part--the big differences relate to how it's "attached externally", e.g., being add-able to Proxy and frozen instances), and then there's some cases that throw a TypeError when things aren't allowed. This case that we spent so long on was an example of throwing such an error.