Dependency on [DOM] due to inheriting from EventTarget

[tobie]

Inheriting from EventTarget (and relying on the event system) adds a huge dependency on the Generic Sensor API which makes it unfit for inclusion in non browser platforms (e.g. node/io).

Actions:

• [x] Define whether ability to include the spec in non browser platforms is a design goal.
• [x] Investigate solutions to lighten that dependency (e.g. future extraction of the event APIs and event loop concept from DOM and HTML into EcmaScript?)
• [x] Seek advise from @domenic who's solving similar problems with the Streams API.
• [x] Study behaviors (ping @kriskowal) and define whether or not their model is more appropriate to expose sensor data than EventTargets.

Proposed resolutions:

• Ability to include the spec in non browser platforms is a design goal but not a deal breaker.
• Change in sensor value will be communicated to the consumer of the API through the EventTarget interface (despite its shortcomings).
• Further work required to identify lower-level primitives. Tracking in #24.

[domenic]

I'm not really sure how to solve this without blocking on some very speculative future work :-/. E.g. EventTarget as-is is almost certainly not going to be ported to ES. (Too many weirdnesses: awkward method naming; bubbles and cancelable and capturing not being generally applicable; the legacy EventListener callback-interface; the Event interface being a big mess; etc.) If something is going to get into ES, it might end up looking like observable, but I have my doubts on that (see issue tracker over there); if nothing else it'll take a long time. And then there's the question of whether a sensor is better modeled as a signal/observable or a behavior...

[smaug----]

Why wouldn't bubbling and cancelable and capturing be applicable in ES? You'd still have some kind of event target path which means bubbling and all that would be needed. (But I'm totally happy to keep DOM Events in DOM world, but that doesn't mean DOM Events couldn't be used outside browsers. DOM events are rather simple. )

[domenic]

In general there is no analogous tree-of-targets structure in ES programs; the DOM is a special case in that sense. Even when there is such a tree-of-targets (or more likely graph), the knobs provided by cancelable, bubbles, and capturing are rarely the ones applicable or desired.

[tobie]

Thank you @domenic, that's precisely the information I was looking for. I'll look at behaviors and might ping you with more questions down the road.

[kriskowal]

@tobie, I think behaviors and observers would implement the iterator interface. next() would pull the current sensor value, or capture the most recently emitted sensor value. For behaviors that do not have a beginning or end, I think it would be reasonable to also implement now(), that would just return the value. Observers would additionally have methods like "forEach", which would call the callback on the stack whenever the underlying sensor emitted a value. I do not think you get away with just one or the other since sensors have different underlying semantics. Continuous sensors that pull data, e.g., temperature, would be appropriate for a behavior. Discrete sensors that push data, e.g., a counter, would be appropriate for an observer. For observers, it is useful to have a distinct observable, to refcount consumers and go either hot or cold depending on whether anyone’s watching. That’s the summary.

This is the most recent observable proposal from the @zenparsing + @jhusain convergence https://github.com/zenparsing/es-observable

[kriskowal]

Also, I agree that it is important not to couple sensors to hierarchical event management. You can always emit events into a DOM, but it’s senseless overhead if you don’t have a DOM to emit into.

[smaug----]

Events don't require a DOM (tree): XHR, WebSocket, EventSource, WorkerGlobalScope etc. And even if you want some propagation, all you need to decide is the path, and the path can be formed also within some graph. There is no requirement for a tree.

[kriskowal]

@smaug, pardon, there is no need to couple any of these things to multiple dispatch, regardless of the topology of event propagation.

[domenic]

While I agree behaviors and observables might be appropriate for many sensors, in general I was trying to caution against taking a dependency on speculative future proposals, rather than encourage it. Just EventTarget is likely the best way to implement this if you want to ship it in a reasonable time frame.

[kriskowal]

@domenic You’re not wrong. Not wrong at all.

[jhusain]

Depending on EventTarget will no doubt be expedient. However it doesn't necessarily make the web a better place given the baggage that it brings. I'm looking for precisely these motivating cases in the web platform to forward my proposal, so I would love to work with you to determine if Observable is a good fit for sensors.

[rwaldron]

Based on Example: Observing Keyboard Events, I don't see how the Observable interface can benefit a potentially high frequency (eg. 1kHz) sensor. Is it really ideal to expect that next() might be called once per ms? Maybe I've misunderstood something there?

[zenparsing]

@rwaldron Purely out of curiosity, what do you think would be an appropriate interface for delivering/consuming high frequency data?

[kriskowal]

@zenparsing Iterator. Consumer calls .next() on demand.

[jhusain]

Can you clarify? Are you concerned about performance? If so, why would calling an event target handler be any better?

The main advantage of using an observable is that it makes it easy to compose multiple sensors together. Developers have to deal with more and more streaming data sources and I think it's imperative that we give them a compositional mechanism for combining them together. In addition to allowing sampling, throttling, and debouncing, observable can declaratively combine sensors together, and declaratively discontinue listening to some signals when other sensor messages arrive.

When I get a chance I'll see if I can put together a simple code example demonstrating the type of composition that I'm talking about.

[domenic]

Composability is not a good argument. Any of these things is just as composable as the other when you add enough methods.

[mattpodwysocki]

@kriskowal couldn't disagree more. Sensors are push data. Why would you force the consumer to call .next()?

[domenic]

Sensors are generally pull (poll), actually, as implemented at the lowest level and in hardware.

[mattpodwysocki]

Yes, but why force that upon the user to have to make that call? Should that be invisible to the user or not?

[domenic]

To be clear: you've switched your argument from "sensors are push data" to "sensors are pull data but users might like push data interfaces more"?

[mattpodwysocki]

No, at the lowest level they may be polling, but it is invisible to the user at most systems, therefore it is seen as push. I haven't changed any of my opinion here

[kriskowal]

@mattpodwysocki If the sensor models a continuous time series, or so discrete as to seem continuous, poll/pull is appropriate. Observables are appropriate for discrete time series data, or data that is sampled at a discrete frequency on your behalf, provided that the sample frequency is granular enough that the consumer would be inclined to poll much more frequently that the value actually changes.

[jhusain]

@Domenic just to clarify, I'm speaking specifically about eventTarget which doesn't have a place to put those methods or the well-defined stream stop semantics on which you can build many different combinators. I'm having difficulty understanding why event target is a good choice given the need to combine sensors together.

I don't argue with the fact that iterators are every bit as composable as observables. It's an interesting discussion which sensors are a better fit for iterators versus observables.

[domenic]

@jhusain we can add methods to EventTarget just as easily (actually more easily, given TC39 process) as we can to Observable.

EventTarget is a horrible interface. But it is already shipping. If composability is an important thing to do with sensors, we should extend EventTarget.

[rwaldron]

Purely out of curiosity, what do you think would be an appropriate interface for delivering/consuming high frequency data?

Based on my experience with JavaScript programs (running in node.js on a variety of platforms) that either read sysfs GPIO directly or via some compiled native bindings, delivering an indication of value change or value read by emitting an event has been both extremely efficient. It's also very intuitive for those writing the programs.

Composability is not a good argument. Any of these things is just as composable as the other when you add enough methods.

Agreed.

"sensors are pull data but users might like push data interfaces more"?

This is actually my exact experience with anyone that's ever used JavaScript to interact with sensory hardware.

EventTarget is a horrible interface.

Yes, ideally a standardized EventEmitter or similar would be preferable.

[domenic]

No, at the lowest level they may be polling, but it is invisible to the user at most systems, therefore it is seen as push.

Right, so it becomes a question of whether we want to expose low-level primitives, or higher-level abstractions. I think the lower level primitives make a lot of sense here (in many cases), because they allow more use cases. For example, note how games use polling of mouse/keyboard and gamepad buttons to achieve lower latency. (I've gotten many complaints about the event-based mousemove API actually at conferences from game developers.) It also allows more conservative or on-demand sampling instead of at a specific predefined frequency. Given that these calls will involve IPC to some degree that can be valuable.

That said,

Based on my experience with JavaScript programs (running in node.js on a variety of platforms) that either read sysfs GPIO directly or via some compiled native bindings, delivering an indication of value change or value read by emitting an event has been both extremely efficient. It's also very intuitive for those writing the programs.

This is actually my exact experience with anyone that's ever used JavaScript to interact with sensory hardware.

from @rwaldron is good in-the-field experience, and we should heed it.

Yes, ideally a standardized EventEmitter or similar would be preferable.

Key word "ideally" :). The main thing I want to communicate in this thread is: don't block your spec on other specs.

[jhusain]

I would definitely prefer an eventemitter to extending event target, but I'd prefer an Observable to both. Doubling down on EventTarget by adding yet more semantics doesn't make sense to sense to me. I also think it's easier said than done.

One nice thing about Observable is that you can declaratively start listening to one sensor when a message arrives from another:

var values = sensor1.takeWhile(pos => pos.x - pos.y > 0).concat(sensor2);

Combinators like concatenation are possible because Observables, like Iterables, have a concrete notion of completion. @domenic Would you add the notion of completion to EventTarget? How would that be communicated? Would unsubscription be implied? Would it integrate is well with existing features like generators as observable?

It would really nice to see something concrete in terms of a proposal about how event target composition would work (if that is indeed what you are proposing). Maybe you could work something up?

[rwaldron]

because they allow more use cases. For example, note how games use polling of mouse/keyboard and gamepad buttons to achieve lower latency. (I've gotten many complaints about the event-based mousemove API actually at conferences from game developers.)

Thanks for bringing this up—it's super important. The current design of this spec understands the above and makes sure that it's well addressed. The value property of any sensor instance is an accessor that returns the most present possible reading of sensor. It starts off as null and once the underlying system has begun reading the actual sensor, it will return those reading values.

let ambient = new sensors.Light({
  // some yet-undesigned way of telling this sensor to synchronize with requestAnimationFrame
});

requestAnimationFrame(function frame() {
  requestAnimationFrame(frame);

  // https://developer.mozilla.org/en-US/docs/Web/API/PowerManager/screenBrightness
  let brightness = calculateBrightness(ambient.value);  
  if (PowerManager.screenBrightness !== brightness) {
    PowerManager.screenBrightness = brightness;
  }
});

One nice thing about Observable is that you can declaratively start listening to one sensor when a message arrives from another:

var values = sensor1.takeWhile(pos => pos.x - pos.y > 0).concat(sensor2);

What exactly is the value of values and for how long?

Combinators like concatenation are possible because Observables, like Iterables, have a concrete notion of completion.

Can you explain how this applies to a sensor that a program is expecting to read at a minimum frequency of 200Hz for the life of the program itself?

Would you add the notion of completion to EventTarget? How would that be communicated? Would unsubscription be implied?

I don't "endorse" it, but removeEventListener would be effective.

[jhusain]

@rwaldron the type of values would be whatever was in sensor 1 until the condition was met and then sensor 2. Let's say accelerometer positions for example. The example is just to demonstrate that declarative composition is possible if the underlying type supports a completion semantic. I'd love to put together a real use case and will do so when I get a chance. Suggestions of thorny signal composition problems are welcome.

If all you are doing is listening to a single sensor for the life of a program and taking no action beyond updating a variable, composition won't be very interesting to you. My suspicion is that there are many use cases for combining sensors into new more complex sensor streams.

Remove event listener is insufficient to describe concatenation. Let me try and do a better job of explaining. An Observable emits a special completion signal (like generator) when it's finished.

sensor.subscribe({ next(v) { /* got a sensor value / }, return(v) { / got a completion signal */ });

A completion signal might seem unnecessary for a sensor, which is presumably an infinite stream. However it is interesting to note that you can compose infinite streams together with functions and other streams to create finite streams. That is what this subexpression does:

sensor1.takeWhile(pos => pos.x - pos.y)

This creates an observable which ends when the condition is met. Observables also free subscriptions when they end, so there is no need to call remove event listener. Because the Observable type has a clear end signal, you can build more interesting combinators like concatenation:

sensor1.takeWhile(pos => pos.x - pos.y).concat(sensor2)

The concatenation method looks for the return method to be called, and then begins listening to the other sensor. Without an explicit completion semantic in EventTarget, building either of these methods isn't possible. We'd have to add more semantics to EventTarget to make that happen.

[rwaldron]

If all you are doing is listening to a single sensor for the life of a program and taking no action beyond updating a variable, composition won't be very interesting to you.

This is most real world use cases. These things are long running and serve only to update and inform some other operation:

• Checking Proximity of face to device to automatically "lock" the screen (so you don't cheek press)
• Any environmental monitor that affects change via output to any kind of effectors (eg. smooth display updates based on any number of sensory inputs, eg. updating the screen brightness by monitoring ambient light)
• Controlling the position of a drone by updating its PID controllers with data from a smart phone's gyro and accelerometer, which are being used as a virtual "joystick" in a remote control app (see: FreeFlight 3, or anyone that works on a modern farm)
• Monitoring moisture in the surrounding environment (don't drop your phone in the toilet)
• Controlling the volume of headphones based on ambient sound level (good for ignoring everyone on crowded noisy subways)
• Monitoring heart rate via a built-in smartwatch sensor
• Ambient temperature monitoring, coupled with an algorithm that uses Compass, Acceleration, GPS and GSM location data, which sends a message to central "smart home" system to adjust where you are to who you are, and readjust/balance to any other user of the same system.
• GPS/Compass/GSM data to track person or vehicle while using turn-by-turn directions
• GPS/Compass/GSM data is also used to make "traffic" display more accurate
• ^^ + Accelerometer/Gyroscope for monitoring a bike ride, walk, jog, etc.
• All of the above for any kind of game.

Here's an example that the sensor.takeWhile form might be useful for:

• Fingerprint validation.

(but can still be done without it)

My suspicion is that there are many use cases for combining sensors into new more complex sensor streams.

Yes, PID controllers, fusion algorithms, etc. but the API doesn't have to more than object.value for those to be easy and awesome to write.

sensor1.takeWhile(pos => pos.x - pos.y) This creates an observable which ends when the condition is met.

What is the condition? Truthy-ness/falsey-ness of the return value?

Without an explicit completion semantic in EventTarget, building either of these methods isn't possible. We'd have to add more semantics to EventTarget to make that happen.

Or not at all because it's not immediately obvious (or even remotely evident) that this is a desirable semantic trait that befits a solution for the most common use cases.

[jhusain]

I respect your experience in this area, and it's great to have these real world use cases to consider. If the main mechanism of composing together signals is indeed through examining variables fed at high velocity that's a great data point. Composition would be more interesting if other asynchronous actions like network requests or animations were ever kicked off/cancelled by signals. Then you would have concurrency to coordinate. However if use cases for composition around sensors are really that rare, developers could always adapt an eventtarget to an observable in those instances. I am concerned about choosing a type to model sensors that does not have a completion semantic, but if you're confident that it is unnecessary that's valuable feedback.

What about error handling? Are there cases in which a sensor stream might terminate due to an error? Is this a marginal case as well?

[rwaldron]

What about error handling? Are there cases in which a sensor stream might terminate due to an error? Is this a marginal case as well?

For these cases, termination usually means something else, eg. battery is dead.

[tobie]

What about error handling? Are there cases in which a sensor stream might terminate due to an error? Is this a marginal case as well?

Seems a simple error event handler fits the requirements here.

[tobie]

Thank you all so much for this fantastic input. Please keep it coming.

Here's what I'm hearing so far:

• Consensus that "EventTarget is a horrible interface", yet… shipping.
• Interest to build on top of Observables and/or Iterators, yet concern about the ETA of both APIs, and uncertainty as to whether such APIs enable real world use cases.
• The value of real use cases and in-the-field experience brought by Johnny-Five. (As a sidenote, I intend do copy pretty much verbatim the use cases into the spec itself.)
• The debate between high level APIs vs. lower level ones and pushing vs. pulling (polling).

My overall feeling is that the current, EventTarget-based Sensor API proposition is what we should go with. The EventTarget dependency is annoying but unavoidable at present. I also tend to think that any successful deployment of Observables will need to come equipped with a story on how to migrate EventTarget-based APIs which we'll be able to hop on.

That said, it might be worthwhile to think about the underlying primitives (e.g. the ones that do the actual polling) and possibly expose them in the spirit of the Extensible Web Manifesto.

[tobie]

Proposed resolutions:

• Ability to include the spec in non browser platforms is a design goal but not a deal breaker.
• Change in sensor value will be communicated to the consumer of the API through the EventTarget interface (despite its shortcomings).
• Further work required to identify lower-level primitives. Tracking in #24.

[gmandyam]

This was discussed on the W3C DAP call on June 11, 2015.

Specifically for the Geolocation I/F and Geofencing API, can an observable (e.g. Promise) be implemented on top of the EventTarget Sensor API without affecting performance?

[tobie]

@gmandyam could you please give code examples of what you would want to see and/or use cases? I'm not sure I understand as EventTarget is an observable.

[gmandyam]

Take the one-time location request example from the existing spec (where a fresh location is required) and show how the Sensor API applies. (Yes - I know this is not a Promise-based API, but it can be adapted to be Promise-based). Note that there are no DOM dependenices in the example as far as I can tell.

// Request a position. We only accept cached positions whose age is not
// greater than 10 minutes. If the user agent does not have a fresh
// enough cached position object, it will immediately invoke the error
// callback.
navigator.geolocation.getCurrentPosition(successCallback,
                                         errorCallback,
                                         {maximumAge:600000, timeout:0});

function successCallback(position) {
  // By using the 'maximumAge' option above, the position
  // object is guaranteed to be at most 10 minutes old.
  // By using a 'timeout' of 0 milliseconds, if there is
  // no suitable cached position available, the user agent 
  // will asynchronously invoke the error callback with code
  // TIMEOUT and will not initiate a new position
  // acquisition process.
}

function errorCallback(error) {
  switch(error.code) {
    case error.TIMEOUT:
      // Quick fallback when no suitable cached position exists.
      doFallback();
      // Acquire a new position object.
      navigator.geolocation.getCurrentPosition(successCallback, errorCallback);
      break;
    case ... // treat the other error cases.
  };
}

function doFallback() {
  // No fresh enough cached position available.
  // Fallback to a default position.
}

[tobie]

Take the one-time location request example from the existing spec (where a fresh location is required) and show how the Sensor API applies. (Yes - I know this is not a Promise-based API, but it can be adapted to be Promise-based).

So I see three things which aren't immediately available here:

1. The ability to lookup a cached SensorReading without polling the sensor, that doesn't seem like a big issue, and could be added in either the Generic Sensor API itself or in the GeolocationSensor. Tracking it here #47.
2. The possibility to stop a SensorObserver before it is garbage collected. In practice I'm not sure that's really needed (Johnny-Five doesn't seem to support it), nevertheless, filed an issue for it: #48.
3. A convenience method to get a single SensorReading through a Promise. This is actually both trivial to spec, implement and even polyfill. It had been provided in a previous iteration of the spec, removed in order to focus on solving lower-level primitives, but could be added back, either in the Generic Sensor Spec itself, or in the Geolocation spec. Tracking here: #49.

The Web developer community would benefit greatly from more consistency across the platform. So I sincerely hope we can resolve those issues and get the Geolocation Working Group fully onboard this effort.

FWIW, you'll note a the new design I'm suggesting in #46 borrows a lot of its aspect (notably the promise-based Sensors.matchAll sensor discovery method) from @timvolodine's initial proposal

Note that there are no DOM dependencies in the example as far as I can tell.

No, but there's a dependency on navigator, thus on HTML, which frankly, is about as bad. ;)

[rwaldron]

Note that there are no DOM dependenices in the example as far as I can tell.

Don't get hung up on this. The only reason that EventTarget is being used is because it's the closest thing to EventEmitter. If that existed, we wouldn't have this problem, because:

var location = new Sensor.GPS();

location.once("change", function() {
  // Located. 
  console.log(location.latitude, location.longitude);
});

[marcoscaceres]

< rant > I'll note how much frustrating effing BS it is that we still don't have EventEmitter on the Web. We should define it.
< / rant >

[rwaldron]

@marcoscaceres honestly, I'd rather see Sensor wait for EventEmitter than continue down the path with EventTarget and all of the baggage it entails.

[rwaldron]

Let's try to avoid conflating concepts: Geofencing is not a sensor in a device, it's an concept that can be implemented in terms of an algorithm in a program which consumes the output of any GPS sensor.

[marcoscaceres]

@rwaldron let's just do it. How is it that fetch() gets to supersede XHR, but we don't get to modernize the EventTarget nonsense?

[rwaldron]

@marcoscaceres completely agreed.

[kriskowal]

A promise would only be useful to distinguish a sensor that has captured a value from one that has not yet. I doubt that an interface that returns a promise for the next captured value would be desirable, but if it is desirable, is trivial to lift from any sane primitive.

Polling a sensor, a sentinel value of null or NaN should be fine to make that distinction. It should be possible for composite sensors to treat these as viral contagions. NaN is viral for free for the domain of numbers.

Providing an interface for counting interested consumers is a different story. An interface that returns a promise for a sensor after it has captured its first value would both express intent to consume and pass ownership of the consumer. sensor.destroy() could communicate a loss of interest.

On Thu, Jun 11, 2015 at 1:42 PM, Tobie Langel notifications@github.com wrote:

Take the one-time location request example from the existing spec (where a fresh location is required) and show how the Sensor API applies. (Yes - I know this is not a Promise-based API, but it can be adapted to be Promise-based).

So I see three things which aren't immediately available here:

1. The ability to lookup a cached SensorReading without polling the sensor, that doesn't seem like a big issue, and could be added in either the Generic Sensor API itself or in the GeolocationSensor. Tracking it here

   47 https://github.com/w3c/sensors/issues/47.

2. The possibility to stop a SensorObserver before it is garbage collected. In practice I'm not sure that's really needed (Johnny-Five https://github.com/rwaldron/johnny-five/wiki/Sensor doesn't seem to support it), nevertheless, filed an issue for it: #48 https://github.com/w3c/sensors/issues/48.
3. A convenience method to get a single SensorReading through a Promise. This is actually both trivial to spec, implement and even polyfill. It had been provided in a previous iteration of the spec, removed in order to focus on solving lower-level primitives, but could be added back, either in the Generic Sensor Spec itself, or in the Geolocation spec. Tracking here: #49 https://github.com/w3c/sensors/issues/49.

The Web developer community would benefit greatly from more consistency across the platform. So I sincerely hope we can resolve those issues and get the Geolocation Working Group fully onboard this effort.

FWIW, you'll note a the new design I'm suggesting in #46 https://github.com/w3c/sensors/pull/46 borrows a lot of its aspect (notably the promise-based Sensors.matchAll sensor discovery method) from @timvolodine https://github.com/timvolodine's initial proposal https://www.w3.org/2008/geolocation/wiki/images/6/69/TPAC-sensors.pdf

Note that there are no DOM dependencies in the example as far as I can tell.

No, but there's a dependency on navigator, thus on HTML, which frankly, is about as bad. ;)

— Reply to this email directly or view it on GitHub https://github.com/w3c/sensors/issues/21#issuecomment-111272750.

[rwaldron]

Polling a sensor, a sentinel value of null or NaN should be fine to make that distinction.

Yep, this is exactly how it's designed.

[tobie]

Polling a sensor, a sentinel value of null or NaN should be fine to make that distinction.

Yep, this is exactly how it's designed.

:+1:

[tobie]

@rwaldron let's just do it. How is it that fetch() gets to supersede XHR, but we don't get to modernize the EventTarget nonsense?

Problem with EventTarget is it's everywhere. A possible strategy might be to define EventEmitter then, in DOM, do:

interface EventTarget : EventEmitter {
  // …
}

and see where that takes us. Of course, we'd just use it directly here.

[zenparsing]

honestly, I'd rather see Sensor wait for EventEmitter than continue down the path with EventTarget and all of the baggage it entails.

I find this a little odd since, from my point of view, EventEmitter and EventTarget are equally terrible user-facing APIs, in that neither one allow for composition and abstraction.

For instance, if you are modeling a push stream and using Observable, then "once" doesn't need to be implemented at the API level; it can instead be implemented as a standard combinator over all observables:

(Taking some liberty with the proposed :: operator.)

function first() {
    return new Observable(sink => {
        let subscription = this.subscribe({
            next(v) { 
                sink.next(v);
                // Stop listening after we've received one event
                subscription.unsubscribe();
            },
            throw(v) { sink.throw(v) },
            return() { sink.return() },
        });
    });     
}

sensor.observe("change")::first().forEach(x => {
    // do whatever, but only once
});

I'm not saying the Sensor API should necessarily be implemented using observables, but I think we should be looking forward to compositional solutions like Observables and AsyncIterators rather than old-school event dispatch APIs like EventTarget and EventEmitter.