How do we request a specific amount of data from phant?

[liudr]

Phant is great for uploading. It's simple and elegant. On the other hand, how about make a simple elegant data retrieval method such as retrieve according to a few criteria:

1. extract from record number (oldest being 0 and newest being -1) for a fixed length of records. This way we can extract records for plotting, such as last 1024 data points. page=1 won't do it. It seems to be extracting the most recent data that are not filling up a page yet, so its length is variable and up to 50K but down to say 1 data point. Having the ability to extract last X data points is essential for plotting.
2. extract according to time stamp of upload. Say extract from time stamp xxxxx and 1024 records since that time stamp. This helps logging data offline. Say you want to download your data, you would be worried that you download the whole thing will waste sparkfun bandwidth or time or having a lot of duplicate data. You would be more interested in download say 10/20-10/30 data and save on your computer for further analysis. It doesn't have to be an internet of useless sensors but could be a teaching and/or research tool for regular data upload and extraction so decent records can be kept. Right now it's just for play (sorry for asking for too much).
3. Here is just a thought. When you request metadata of a stream, there should be some custom name=value pairs, that are only kept until it has been requested once, and then erased. It's like a messaging system, where one party posts a message and the other retrieves it, at which time the message disappears. This way I can use one client to set a name=value pair for the data logger to retrieve later. The pair could be "delay_minutes=15" so that when the data logger retrieves it, it alters its data collection period to 15 minutes. Without this, the data logger is just a passive thing and can't be interacted with. I don't think that is most useful in many cases. I use this on some projects to tell loggers deployed in remote areas that it's time to restart or turn on heat to brave the cold :) Thanks for your time!

[toddtreece]

1) The offset & limit params should take care of this. Check out: https://github.com/sparkfun/phant/pull/85 2) Check out: https://github.com/sparkfun/phant/pull/90 for filters & https://github.com/sparkfun/phant/pull/91 for timezone conversion. The timestamp field can be queried using strings similar to PHP's strtotime. i.e. ?gt[timestamp]=now%20-10%20minutes&timezone=America/Denver 3) Great idea, but what about something like this: Currently streams have metadata (stuff like title and description) & logged data (what you see when you request output). What if I allow users to create/update/delete their own custom metadata values, and keep that separate from the logged data so they can use the metadata to control devices if needed. My gut says that the key/value expiring is probably something that should be handled by the client with a delete request, but I could be convinced otherwise.

[liudr]

Thank you Todd! I am finding #85 very useful already. Will read #90 and #91. About the user-defined metadata, allowing users to create/update/delete will be awesome! I wonder as a data logger sending data, how to get the metadata, by requesting metadata or by looking into HTTP response header? I'm pretty new to this stuff.

[toddtreece]

I'm guessing the metadata stuff would be a different request, and it could have its own REST interface.

[liudr]

Thanks. Will metadata stuff be available some time soon?

[liudr]

Another thought: For the meta-data pair issue, we can also make a separate stream for commands. Maybe the user posts a command to this stream and the logger periodically retrieves the latest command to compare with time stamp to decide whether to execute it.

[stil190]

I am new to GitHub. Collecting some data with my Arduino and Phant. Seems this thread answers some of the questions about what I would like to do. However I am not able to access any of the examples. Any suggestions how to see the examples in thread #85?

[liudr]

stil190,

Do you mean an http address sample such as

http://data.sparkfun.com/output/YGv7gG4M6RfwY8d8LapN.csv?limit=10&sample=4

That's one of my streams. You're welcome to take a look but please create your own stream for more testing with #85 issue related options. I'm using this stream quite frequently.

[stil190]

Thanks

I would like to see how you put the filters (ie... Limit) into the Ajax statement. I am unable to open the links in 85.

Hope this makes sense. Thanks for ur reply.

Sent from my iPhone

On Dec 11, 2014, at 7:28 PM, liudr notifications@github.com wrote:

stil190,

Do you mean an http address sample such as

http://data.sparkfun.com/output/YGv7gG4M6RfwY8d8LapN.csv?limit=10&sample=4

That's one of my streams. You're welcome to take a look but please create your own stream for more testing with #85 issue related options. I'm using this stream quite frequently.

— Reply to this email directly or view it on GitHub.

[liudr]

You mean ajax calls like this?

        var jsonData = $.ajax({url: 'https://data.sparkfun.com/output/' + public_key + '.json',
            data: {limit: 10, sample: 4},
            dataType: 'jsonp',
        })

Replace public_key with your own public key.

[stil190]

Thanks.  That is exactly what I was needing to understand.  Appreciate the help. From: liudr notifications@github.com To: sparkfun/phant phant@noreply.github.com Cc: stil190 shepherd_joe@yahoo.com Sent: Thursday, December 11, 2014 11:24 PM Subject: Re: [phant] How do we request a specific amount of data from phant? (#126)

You mean ajax calls like this? var jsonData = $.ajax({url: 'https://data.sparkfun.com/output/' + public_key + '.json', data: {limit: 10, sample: 4}, dataType: 'jsonp', }) Replace public_key with your own public key.— Reply to this email directly or view it on GitHub.

[bboyho]

Phant is No Longer in Operation

Unfortunately Phant, our data-streaming service, is no longer in service and will be discontinued. The system has reached capacity and, like a less-adventurous Cassini, has plunged conclusively into a fiery and permanent retirement. There are several other maker-friendly, data-streaming services and/or IoT platforms available as alternatives. The three we recommend are Blynk, ThingSpeak, and Cayenne. You can read our blog post [ https://www.sparkfun.com/news/2413 ] on the topic for an overview and helpful links for each platform.