Data sources other than google docs spreadsheet such as raw CSV or JSON files (or APIs)

rufuspollock commented 10 years ago

If CSV or JSON source supports CORS this is very easy to do since we just need to swap out "backend" in data package json for each timeline (CORS matters because the current data load actually happens in JS frontend ...)

If you want CSV / JSON without CORS we have to make some more fundamental changes - essentially node app has to proxy stuff in ...

RouxRC commented 10 years ago

+10, this is really a bummer to be forced to rely on google when using a free software tool :/

rufuspollock commented 10 years ago

@RouxRC understood though not everyone can edit CSVs and paste online easily - and at least atm you needed something CORS compliant which was an issue.

This is pretty easy to fix and has been requested a few times so should be quite easy ..

PS: if you liked the tool please consider making a donation to the Open Knowledge Foundation http://okfn.org/support so we can keep making stuff like this :-)

betacoder commented 10 years ago

Yes This Feature is a Must , because of Double Talk Privacy , Google can be least trusted with our Private data.

I think the csv copy and paste feature and later turning into json file would solve the issue.

Demo of CSV to json converter online - http://shancarter.github.io/mr-data-converter/

simonacca commented 10 years ago

+10 JSON support will be great. With that timemapper could be incorporated in other web apps that generates their content dynamically.

rufuspollock commented 10 years ago

All: this is actually super-simple to do - it just needs some to hack it in :-)

I would note talk re privacy seem a bit off here since all the data is public by necessity ;-)

Spencerx commented 10 years ago

Basically supporting RDF/XML, you should be able to support common formats such as ATOM, RSS. Microdata support would help when parsing metadata. CSV and XML/RDF go hand in hand so supporting both would go a long way in the right hands.

romeozi commented 9 years ago

Hi there, I'm thinking about put the hands on timemapper (if you didn't plan it yet ;) ) to use a different backend data source ( json, csv). Could you please point me the right way to do it?

rufuspollock commented 9 years ago

@romeozi there are 2 options

Option 1

simplest option is really easy - just change the resource info in the DataView when we save. To illustrate, here's the JSON object we store and how we would change it:

http://timemapper.okfnlabs.org/api/dataview/rufuspollock/major-battles-napoleonic-wars

{
  "name": "major-battles-napoleonic-wars",
  "owner": "rufuspollock",
  "title": "Major Battles in the Napoleonic Wars",
  "licenses": [
    {
      "id": "cc-by",
      "name": "Creative Commons Attribution",
      "version": "3.0",
      "url": "http://creativecommons.org/licenses/by/3.0/"
    }
  ],
  "resources": [
    {
      // change backend to e.g. csv
      "backend": "gdocs",
      // we can't put direct url of CSV as will not support CORS - so use http://datapipes.okfnlabs.org/ to proxy CSV with CORS support
      "url": "https://docs.google.com/a/okfn.org/spreadsheet/ccc?key=0AqR8dXc6Ji4JdEpzNGxLY3FpQjJtdDFkNkVIUlZ2cnc&usp=drive_web"
    }
  ],
  "_last_modified": "2013-10-13T16:44:28.828Z",
  "_created": "2013-10-13T16:44:28.828Z"
}

Option 2

this would involve the node app taking care of proxying the data in to the TimeMapper JSON object (atm the frontend js takes care of loading the data from the url that is given). This is probably cleaner and better longer term (it also allows for caching and other nicer things).

What would happen here is:

still store url and type (e.g. csv, gdocs,
timemapper node app takes care of loading the data from that file into a decent json structure (see e.g. https://github.com/okfn/timemapper/blob/master/public/js/view.js#L207)
that data is passed in with rest of TimeMap object (i.e. that JSON) above
then the frontend is much simpler - it just takes that data it is given and renders it ...

Both Options

You need to change create interface to allowing specifying the type of the file (csv, json, ...)

Nolski commented 7 years ago

It looks like this repo is no longer maintained but I'm fairly certain that since timemapper just uses recline, you can modify the recline model to take a json list of records that are in the format that timemapper expects. I'm updating an old project that uses timemapper to pull records from a database and will report back with a PoC once i get one running.

rufuspollock commented 7 years ago

@Nolski we're still around - and we'd love to do more but it would take a little bit of resourcing from somewhere or an enthusiastic volunteer to make some progress.

Nolski commented 6 years ago

Hey @rufuspollock, great to hear! Maybe I can help out a bit then :). So since timemapper is just based on recline (I'm a bit new to it so forgive me if I'm stating anything obvious), getting timemapper able to use json objects instead of google spreadsheets was pretty simple.

When the recline model dataset is created, all you have to do is pass in an array full of your timemapper events and be sure to set the fields property to be all of the keys in the objects. I named the object keys the snake_cased version of the names you use in the google docs template and it all seemed to work just fine.

I have a working example here. https://github.com/Nolski/mordecai-cms/blob/master/static/js/view.js#L6

rufuspollock commented 6 years ago

@Nolski great! If you want to create a test we could get a pull request in here and have added to the platform.

okfn / timemapper