Revising GET /input

[jasonkuhrt]

https://github.com/GoogleCloudPlatform/kubernetes/pull/1318

Hm, this got me thinking.

What if we kept a small back-log of cloudBit input that users can either tail or follow? By default they would get “the past” (pull), but if they wanted “the present” (push) they could choose to follow.

We could base the back-log on 1) time or 2) quantity or 3) a default + user can choose.

In addition to any genuine use-cases we agree on for this (TBD) we also suspect that novice users may have a hard time with streaming because simple as it is, relatively poor mainstream techniques/knowledge/tooling/teaching around dealing with the dimension of time in programs makes it feel harder than it actually is. For example see #1

One reservation I have is that I don't want to facilitate polling whatsoever. For us polling is comparatively (to streaming) a strain on our system and for users the performance sucks, which in turn for us makes our product seems weaker than it is. We have to craft something that helps people "fall into the pit of success" as Pete Hunt likes to say about ReactJS.

So, for example, we might arbitrary throttle repeated tail GETs from the same client and even return an error at some point. Imagine if we detect more than three tail GETs within ten seconds, we could return information containing this point:

{
message: "Whoops, it looks like you are polling. Our platform by design
does not support this. If you really want to follow the input of your cloudBit
then you should use this query param: `follow=true`. 
Our server will return a stream (using standard Server Sent Event
technology) that your program can tune into. Learn more and try some
interactive demos here
developers.littlebitscloud.cc/learn/api/http/input#streaming.",
...
}

[jasonkuhrt]

CC: @stresslimit @alin-littlebits @NelsonMinar @manitou48

[NelsonMinar]

I think it's a good idea to provide a simplest-possible API for getting data off the device. When I wrote my comments in #1 it was because I was trying to set up a simple remote temperature sensor (see below). Polling every 10 minutes was exactly right for my application. Don't even need a fancy tail for that, just the last reading, and I was hoping a simple wget would do it.

But I totally get your point that polling can be bad and you want developers to understand the power of the data streams from the beginning. I'm not quite sure how to do that other than documentation. If you do offer a polling interface you will definitely want to throttle it somehow.

[jasonkuhrt]

Polling every 10 minutes was exactly right for my application

Perfect, and this is a definition of polling that I think we should support (side note: We should maybe be careful about wording here. Anything can be polled, maybe the gist we need to communicate is that we are against "real-time polling").

@NelsonMinar I don't know what your client was but say it was a very low power/incapable device, it might not even be possible to consume a real-time data source. Sure, it would be nice if everything was real-time but the reality is there exists lots of use-cases where real-time is actually not desirable for e.g. power reasons. UX is not the only user concern.

[hyperobject]

This makes a whole lot of sense especially in the context of my Python library - using the stream for realtime data is really elegant in Python

for pct in bit.stream:
    print pct

but streaming-only pretty much eliminates any other simple use-cases (command line tools, irc bots (input only needed when someone says the command word), etc).

One other question though, is the current streaming data JSON formatted? Python keeps throwing errors when I try to decode cloudBit input with the standard json lib.

[manitou48]

the streaming protocol has an extra data: at the front of the JSON data

On Tue, Jan 13, 2015 at 03:54:44PM -0800, Technoboy10 wrote:

This makes a whole lot of sense especially in the context of my Python library - using the stream for realtime data is really elegant in Python

for pct in bit.stream:
    print pct

but streaming-only pretty much eliminates any other simple use-cases (command line tools, irc bots (input only needed when someone says the command word), etc).

One other question though, is the current streaming data JSON formatted? Python keeps throwing errors when I try to decode cloudBit input with the standard json lib.

---

Reply to this email directly or view it on GitHub: https://github.com/littlebits/cloud-platform/issues/2#issuecomment-69845600

Tom Dunigan thdunigan@gmail.com http://tnlandforms.us

[hyperobject]

@manitou48 Ah, ok. Thanks. :)

[stresslimit]

So the input endpoint would NOT have the SSE stream by default, only if you added a GET param? For example:

GET /devices/{id}/input will get the value once then close the http connection

GET /devices/{id}/input?follow=true will return the SSE http header and stream the response

I like this. Thoughts?

[manitou48]

I like it.

On Wed, Jan 21, 2015 at 01:51:02PM -0800, Colin wrote:

So the input endpoint would NOT have the SSE stream by default, only if you added a GET param? For example:

GET /devices/{id}/input will get the value once then close the http connection

GET /devices/{id}/input?follow=true will return the SSE http header and stream the response

I like this. Thoughts?

---

Reply to this email directly or view it on GitHub: https://github.com/littlebits/cloud-platform/issues/2#issuecomment-70928889

Tom Dunigan thdunigan@gmail.com http://TNlandforms.US

[Mahjouba91]

Hi, no news about this subject ?