Agents, vats, continents, oh my: Rephrase agents properly in terms of job queues and realms

[lars-t-hansen]

Currently this is specified only in a terminology entry (since the creation of an agent is outside the spec).

However, an agent has its own initial job queue and initial realm, cf E262 8.5. This will warrant more than a terminology entry, but it will probably be little more than a variation on the existing 8.5, as the agent really is an independent ECMAScript instance.

[lars-t-hansen]

So if an agent maps to an "event loop", each worker is an agent, whether it meets the forward progress guarantee or not. Is that problematic?

Not yet, I think. The forward progress guarantee stands and has implications for the embedding. As I argued above, I think even a shared-thread event loop can provide forward progress guarantees if agents that share a thread have CanBlock=false.

Though perhaps we could solve this when we add "dedicated thread". Dedicated thread would be the feature that guarantees your own event loop, whereas otherwise workers may share an event loop (which is what at least Firefox seems to be doing at times). Is there an issue yet on "dedicated thread" workers?

No... Let me be clear. The shared memory spec does not need an extension to workers with a dedicatedThread option; it will simply require workers to run on a dedicated thread, and if the embedding makes no provisions for distinguishing dedicated and non-dedicated threads then the embedding will basically be forced to use a dedicated thread for the agent no later than when the agent starts sending or receiving SharedArrayBuffers. I'll file a whatwg issue this morning suggesting the feature, and I think it's a very nice idea, but it probably won't be high priority with me to push it along.

[annevk]

Then the embedding will basically be forced to use a dedicated thread for the agent no later than when the agent starts sending or receiving SharedArrayBuffers.

This can be done dynamically?

[khuey]

Not in Gecko. On Mar 11, 2016 10:07 PM, "Anne van Kesteren" notifications@github.com wrote:

Then the embedding will basically be forced to use a dedicated thread for the agent no later than when the agent starts sending or receiving SharedArrayBuffers.

This can be done dynamically?

— Reply to this email directly or view it on GitHub https://github.com/tc39/ecmascript_sharedmem/issues/27#issuecomment-195379193 .

[annevk]

Right, so maybe @lars-t-hansen meant it as a "not my problem" declaration? It seems either all workers will have to guarantee forward progress or we need to introduce this constructor option, there's no middle ground. (And if we introduce this constructor option to avoid changing how workers work today we should probably require it to be set before making SharedArrayBuffer works, otherwise you end up with the weird semantics of sometimes allocating a worker you can share memory with, and sometimes not.)

[lars-t-hansen]

Well... I was really intending to note that an implementation could in principle perform this kind of resource management behind the scenes until a SharedArrayBuffer is shared from it or to it. Indeed, if what @sicking wants comes to pass -- having a pool of M threads to run N > M workers in some fair fashion -- then making the determination dynamically does not seem like a stretch.

Also, @annevk, I'm very confused by your remark about the constructor option, perhaps you can clarify. Whether a thread can block is orthogonal to whether it has access to shared memory. You should always be able to share memory with one of your dedicated workers (other kinds of workers, not so much, of course). But that worker would really only be allowed to block if it has a dedicated thread to execute it.

EDIT: But for sanity's sake, all dedicated workers would effectively need dedicated threads, once they have sent or received shared memory, because we want them to be able to block on cells in that memory.

EDIT 2: I suppose that in principle it would be possible to promote a thread from shared to dedicated once an agent blocks. In effect, blocking on a shared thread would add a new thread to the shared thread pool and take the current thread out of it.

[annevk]

Thanks for the clarification, I'm still struggling with understanding the various shared memory constraints, so I might confuse them now and then.

[lars-t-hansen]

I think this can be closed now.

The matter of terminology has been resolved.

This PR defines agents and forward progress in ES262 almost without reference to the shared memory spec:

• https://github.com/tc39/ecma262/pull/522

There are a couple of whatwg bugs that reference related issues about dedicated threads and spurious termination:

• https://github.com/whatwg/html/issues/851
• https://github.com/whatwg/html/issues/1004

The rest of the missing pieces we can file as new tickets, as we encounter them.