v1.0 release discussion

[rufuspollock]

@andylolz @davidmiller @rossjones (and all others!)

We've now done a lot since the original prototype and even 0.1 "announce" and are starting to get something really quite nice (in large part thanks to efforts of @davidmiller and esp recently @andylolz!).

This is a discussion ticket on what are the priority things to do to make this really super awesome and ready for a v0.2 (or v1.0!). To get the ball rolling here's some suggestions:

• [ ] Arbitrary maps #21 - this would really be something novel and very cool (i especially like the gist it option @rossjones has suggested) e.g. call the op gist and have /gist {id}/ ...
• [x] Command line #19 - we're nearly there - this just needs some polishing
• [ ] Node library #18 (mostly done but we'd need some docs)
  • Do we want to have datapipes-web or something if datapipes is core library plus cli (?)

General point (not a feature)

• [ ] Documentation / tutorials #85 (a bit more here would be really useful - perhaps a series of short blog posts)
• [ ] Promote this relevant places (perhaps a guest post here or there ...) - perhaps get some big data juice by going on about how we can handle GB csvs because of our love node streaming :-)

Some non-essential extras

• [ ] dedicated viewacsv app (perhaps separate app from this at viewacsv.okfnlabs.org with a view dedicated options)

[andylolz]

Arbitrary maps

Definitely. I think I have a good idea how to tackle this one now. Just need to get round to doing it!

Do we want to have datapipes-web or something if datapipes is core library plus cli (?)

Are you talking about git repos, or node modules? (or both?) I’m afraid I’m not really sure how the world of npm actually works…!

how we can handle GB csvs because of our love node streaming :-)

This would be great, but means we would defs need to close #15. Would be really good to do that for v0.2 anyway.

dedicated viewacsv app

Would be cool. I guess this relies on creating the datapipes node library first. Then we can create a new app that adds this as a dependency.

[andylolz]

At the risk of pulling loooads of stuff into spec for v0.2… It would also be really good to make the switch to binary-CSV (#93).

In fact, if we did all of these things, I think v1.0 may indeed be more appropriate.

[davidmiller]

Also - support for a second input format. Whichever that may be.

On 5 December 2013 13:50, Andy Lulham notifications@github.com wrote:

At the risk of pulling loooads of stuff into spec for v0.2… It would also be really good to make the switch to binary-CSV (#93https://github.com/okfn/datapipes/issues/93 ).

In fact, if we did all of these things, I think v1.0 may indeed be more appropriate.

— Reply to this email directly or view it on GitHubhttps://github.com/okfn/datapipes/issues/102#issuecomment-29898589 .

Love regards etc

David Miller http://www.deadpansincerity.com 07854 880 883

[andylolz]

Also - support for a second input format

This would mean that the currently trending “viewacsv” could be extended to “viewanrss” (or “viewajson” or whatever).

[davidmiller]

For example, having this work: [http://datapipes.okfnlabs.org/json/head -n 5/html?url=https://api.github.com/repos/okfn/datapipes/issues](http://datapipes.okfnlabs.org/json/head -n 5/html?url=https://api.github.com/repos/okfn/datapipes/issues)

[andylolz]

Actually you know… outputting to json or rss would be straightforward. Does anyone want csv converted to rss?? csvfeeds.com?

[rufuspollock]

@andylolz how are we doing here? is there anything i can / should hack on? What items should we really aim to close? I think a v1.0 / v0.2 release early in new year would be cool!

[rufuspollock]

Command line stuff basically done in dc7ea49b8a9c4bcdb339e7165bba4574c7073350

[rufuspollock]

Renamed this issue to v1.0 discussion from v0.2 discussion ...