search

datasets / country-list

List of all countries in the world with their ISO 2 digit codes (ISO 3166-1) as CSV and JSON
http://datahub.io/core/country-list
150 stars 185 forks source link

Include JSON in repository? #5

Open jochakovsky opened 8 years ago

jochakovsky commented 8 years ago

http://data.okfn.org/data/core/country-list makes both CSV and JSON formats available for download, but only the CSV is directly available in this repository. Would it be possible to include the JSON in this repository as well? Thank you!

rufuspollock commented 8 years ago

@jochakovsky what's the exact use-case?

In general, we want to keep this is a clean tabular data package which means CSV only.

However, we also want to support user needs so good to know the requirement :-)

Glutnix commented 8 years ago

A popular package such as https://www.npmjs.com/package/country-list could really do with getting its dependancies from here.

fannarsh commented 8 years ago

Currently the npm package country-list is getting the data from here, but it needs to convert it from cvs to json.

rufuspollock commented 8 years ago

@fannarsh really useful data point. How are you getting this data package? Are you submoduling it, puling it direct from raw or getting it from data.okfn.org/data/country-list? I note the latter already has a JSON version via the API but it sounds like having json would be useful.

Let me know a bit more about what you'd like and let's see if we can get it working for you 😄

fannarsh commented 8 years ago

Currently I'm pulling it once (per update) from raw and then converting it to json and storing it in my repo. To be honest, whether the data is in csv or json doesn't matter much to me, but of course it would be nicer and a little bit more convenient to be able to download json directly.
On the other hand if you would publish the data as an npm package then I could simple require that package as an dependency and never worry about needing to update the data myself. And that would also make it easier for other developers to include your data in other projects (if there are such developers that wouldn't like to use my module 😀 ).

rufuspollock commented 8 years ago

@fannarsh that's really useful clarification.

And the idea about building to a npm package is a really nice idea. If we did to npm package i assume you'd want it as JSON or would CSV work (or both)? And would it literally just be the raw data in there (plus datapackage.json)?

fannarsh commented 8 years ago

I would just add the JSON, and I wouldn't bother adding the datapackage.json either since its my understanding it's metadata about the CSV data/structure. If I would need to do any edits to the data or bigger work I would probably clone the repo and work from that. The npm package itself would never be a source for doing actual editing work on the data.

fannarsh commented 8 years ago

I added a pull request with an npm package definition that would be good enough for my use case. https://github.com/datasets/country-list/pull/6

rufuspollock commented 8 years ago

@fannarsh reviewing the PR and thank-you for this

Just wondering atm about whether we want in this repo or in a separate repo - we may not want node stuff in here but rather in a small separate repo.

rufuspollock commented 8 years ago

@fannarsh just to say i'm working on this - classic coder issue of trying to make something generic to generate these automatically. If i don't get this sorted soon I'll just take your version and post ...

fannarsh commented 8 years ago

hehe, no sweat, I recognise that problem :)

rufuspollock commented 6 years ago

@fannarsh @jochakovsky we have something to check out now - a npm/node branch in this repo and a published package on npm

https://github.com/datasets/country-list/tree/npm

https://www.npmjs.com/package/@datasets/country-list

It would be great to get your feedback and thoughts here, especially as going forward we are committed to doing this node packaging for more and more of the core datasets. For example:

Any other comments or thoughts warmly welcome.

Aside: we have never forgotten about this. It has just taken a crazy long time for various reasons including some classic yak-shaving: we've been doing a major reboot of https://data.okfn.org and https://datahub.io/ -- which have merged together. Part of that is being able to do a lot of automation ranging from generating the json to generating node packages from data packages ...).

fannarsh commented 6 years ago

Hi @rufuspollock,

I like what you guys have done so far.

I think that you should keep the packages under @datasets org and I like the idea of providing a minimal api like you have done with @datasets/country-list. However I would like to see another package that would be data only, could be named @datasets/country-list-data so that f ex. me could require only the data and keep my package up to date in a easy way. The @datasets/country-list package could even 

// https://github.com/datasets/country-list/blob/npm/index.js
let countryList = require('@datasets/country-list-data');
// instead of 
let countryList = require('./data.json');

But that all depends on how you want to maintain the packages/repos.

If you would not want to release a pure @datasets/country-list-data package then I would suggest adding data exports to the api so that I could access the data and use directly.

rawData = countries.data()

Regarding other datasets, I would say bring them all on 😄 but in reality, workwise it would maybe make sense to start of with *-data packages without the minimal api since that could take more time to figure out. But if the *-data packages are out there then it allows other developers to pick up the thread and do something useful/fun with the data. And churning out packages containing the datasets could just be a question of right tooling.

rufuspollock commented 5 years ago

@fannarsh cool and really useful feedback. Would you like to contribute here - we could give you perms. Also folks on our team like @zelima and @svetozarstojkovic can provide support and guidance :smile: