search

tc39 / test262

Official ECMAScript Conformance Test Suite
Other
2.41k stars 468 forks source link

Why is the Dynamic Code Brand Checks proposal not testable in test262. #4229

Closed ioannad closed 2 months ago

ioannad commented 2 months ago

According to the proposal repo's README,

testing these require affecting the behavior of host callouts so will probably need to be specified as web-platform-tests.

As suggested by @ljharb, this issue is to document the reasons this is not testable in test262, so we can reference this in the tc39/proposals in the place of tests or a testing plan for this proposal.

cc @nicolo-ribaudo

nicolo-ribaudo commented 2 months ago

The reason that this proposal is not testable is that it is just allowing hosts to relax a language requirement.

There may be objects that, when passed to eval, cause a host hook to be called and its result to be used as input to eval.

Even if we required hosts to expose one of such objects in test262, we would be testing a tautology: "to run this test, you must expose an object that when passed to eval it's causes the evaluation of another given string. Then, we test that when we pass this object to eval it evaluates the given string".

ioannad commented 2 months ago

Just for reference, in the above mentioned proposals PR @lukewarlow mentioned the status of WPT tests for this proposal:

https://wpt.fyi/results/trusted-types?label=master&label=experimental&aligned&q=eval this shows the various test files that test TT with eval.

https://github.com/web-platform-tests/wpt/blob/master/trusted-types%2Feval-csp-no-tt.html this is probably the one closest to testing the pure tc39 aspect of this.