What is the most effective way to add 3rd party libraries

[schreiaj]

In my attempt to add plotting to let folks see the data over time better I was trying to add in D3.js and Rickshaw.js (and it's supporting css) I noticed that all of the css and javascript in this project is concatenated into one file. Makes sense during production but it makes adding in new libraries and maintaining the ones in it a complete pain.

What would you suggest the most effective/maintainable method to bring them in is?

[toddtreece]

Good question. I've been using bower to manage client side dependencies, and I have a build script in .bin/ that handles minification and concatenation.

Once you add the js files to .bin/build, you can then run grunt dev to start the dev server, which will watch and automatically rebuild the js and less files whenever they are changed.

The workflow works for me, and I like cutting down the number of http requests when possible, but I'm definitely open to suggestions.

[schreiaj]

Could you use a grunt task to concat and minify when run in production?

In the mean time I'll use the script in .bin/ to get the visualization stuff in.

[toddtreece]

Yep. You could, but then you'd need to move grunt and all of its tasks from the devDependencies list to the dependencies list, which adds a bunch of install time during a deploy. This is why I rolled my own build script for use in dev and production. The extra 20+ package dependencies weren't worth adding for that simple task, and since it's rarely changed, it isn't hard to maintain.

One thing that has stopped me from adding graphs directly to the stream page is that a lot of the data posted isn't meant to be displayed in graph form. People are using it to track their favorite git repos, random number storage, etc. So graphs are probably going to have to be opt in.

I was originally thinking about making a graph generator. You'd give it the stream and the fields you want to graph, and it would generate the JS for you to embed in your own site. If you've got another solution, I'd love to check it out. Thanks for the help!

[schreiaj]

Something like http://bl.ocks.org/schreiaj/raw/09da918ba737ea1f3d66/index.html#dZ4EVmE8yGCRGx5XRX1W ? Changing the hash lets you specify I different stream.

The chart I have implemented handles non-numeric data by simply returning a zero and rendering nothing. I set it up as a separate route with a whole other page. It's more or less the above charting solution with a bit smarter error handling. I'll get a branch up tonight when I get home.

[toddtreece]

That's a great start! What about feeds that have numerical data that isn't meant to be graphed? I'm thinking of things like GPS coordinates. They appear like data that could be graphed, but it really doesn't make sense to display a line graph of latitude and longitude automatically.

I'm not trying to be overly critical of the idea. I went down this same path for universal graphs, and all of my attempts made bad assumptions about the types of info people were logging. I think the best solution might be allowing users to create a graph with a small dialog, and saving their graph config with the stream metadata. What do you think?

[brennen]

It does seem like it'd be really nice to allow specifying graph config. If we figure out microformat annotations, a given microformat could come with a pre-defined graph config...

[schreiaj]

@toddtreece I hadn't thought about lat/lon pairs. I'm just heavily focused on getting visuals put in here due to my belief that the human brain is absolute crap at seeing data change over time using just a column of numbers.

@brennen I do think that the microformat data could be helpful for plotting. Even if it was just a mapping of col:data type. Where data type is things like string, numeric, date, and geo. Stuff similar to selecting the type of data in tools like Tableau.

[toddtreece]

@schreiaj Yeah. It's always been on the list of things to do, I just have been trying to stabilize some other things before looking into it. I wrote a tutorial last week hoping that would help people if they needed it right away, but it's not the long term solution.

I think the solution is probably going to have to require some form user configuration, and I will help integrate the chart config storage if you end up creating a UI that needs it.

[bboyho]

Phant is No Longer in Operation

Unfortunately Phant, our data-streaming service, is no longer in service and will be discontinued. The system has reached capacity and, like a less-adventurous Cassini, has plunged conclusively into a fiery and permanent retirement. There are several other maker-friendly, data-streaming services and/or IoT platforms available as alternatives. The three we recommend are Blynk, ThingSpeak, and Cayenne. You can read our blog post [ https://www.sparkfun.com/news/2413 ] on the topic for an overview and helpful links for each platform.