Planning "Open this in another app" (export) feature

[jywarren]

This can be a list of services we make it easy to export your data to for the next step!

• https://codap.concord.org/releases/latest/static/dg/en/cert/index.html
• https://databasic.io/en/wtfcsv/
• Google spreadsheets
• Plotly?
• Jupiter notebook?

[IshaGupta18]

Yes! I'll explore them up a little and try and implement what is possible!

[IshaGupta18]

@jywarren could you please shed some more light on these options above. In my proposal, I could only explore Google Sheets API. I have looked up the rest right now but I am a little clueless on how we will be implementing them. Could you please share some examples or tutorials or some more information on these? I really liked jupyter notebook. I think plotly will be like chart.js. What do you think?

[IshaGupta18]

Hey @jywarren I looked up a lot of stuff for doing exports. What I found out was that it may not be as simple or direct for CSV files. What we can do is, since we have the parsed data and everything now, we can make a JSON out of it, and this JSON can be used for the same purpose, like here http://tabulator.info/ or https://www.dynatable.com/ .

[IshaGupta18]

What do you think? Converting to JSON is not a problem, my only concern was, is it okay to do this computation? Would it be okay to do this transformation from the 2D array to JSON?

[IshaGupta18]

@namangupta01 what do you think about this?

[IshaGupta18]

And I also think that we need to explore the Google Sheets API a little more as well because we might not be able to trigger it directly, that too requires some more research.

[IshaGupta18]

This is also very interesting! SheetJS would let the user create an excel sheet out of a JSON or a 2D array. https://www.youtube.com/watch?v=41rOAt-zCu4 (sorry for spamming like this but I think export options are very important too)

[IshaGupta18]

Okay, so anyone has any ideas here, I would love it if we could discuss some feasible options here and start implementing them one by one, as they will be direct to implement here. @jywarren @namangupta01 @Souravirus @gauravano @sagarpreet-chadha @rexagod

[IshaGupta18]

@IgorWilbert @geekychasser what do you think of some of the ideas here?

[jywarren]

Yikes, i'm sorry i missed this thread @IshaGupta18 -- catching up now. OK, so the reason I cited a few of these was that they have some interesting analysis capabilities -- that could be useful to fulfill the question you brought up on the call today. That is - "how do we get more information or analysis out of the data?" - as you articulate in https://github.com/publiclab/simple-data-grapher/issues/30 - so, I'm thinking that, very roughly, we might have some kind of set of options like this:

What do you think of something like this?

As to the formatting, I think youre right that some exporting paths may require some data conversion first. But lets start by identifying ones that do not require this. So ways to do exporting might include:

1. Simplest: ability to open a URL like https://example.com?csv=https://publiclab.org/path/to/data.csv
2. Not much more complex: ability to make a POST request that contains the CSV data, to a URL like https://example.com/data/post (or something) using jQuery -- $.post('https://example.com/data/post', { csv: "1,2,3,4..." } -- OR say, just submit a POST request when the user clicks a button, so we wouldn't need jQuery...
3. Requires conversion: here we'd need to run a local function to convert it to the right format, but could then send it as in 1 and 2, like, for example: $.post('https://example.com/data/post', { json: csvToJson(data) }

These won't be straightforward for many potential export types. But we should categorize and sort the options we want to pursue, do it for a few simple ones as a proof of concept, and this can also make it clear to the developers of other projects how we want to be able to integrate.

[IshaGupta18]

Yes! I agree with you here @jywarren , I would love to make a menu panel like this one but as you said, many of the export options aren't as direct as they can be. So, I think that we need to, like you said, sort out the viable options and start implementing them here. Do you have any particular idea of which one should be picked up to start with? Thanks a lot!

[jywarren]

I think Codap might be good: https://codap.concord.org they already have a popup that prompts you to upload a CSV (see below):

I've left a comment on this issue briefly explaining our goal and how we might like to do this: https://github.com/concord-consortium/codap/pull/144#issuecomment-499232827

I think we could open similar comments on other projects to ask how best to do this. It may require some research as well: https://codap.concord.org/for-developers/

https://databasic.io/en/wtfcsv/ is made by folks I know, and they've expressed interest in this functionality before as well: https://github.com/mitmedialab/DataBasic/issues/6#issuecomment-336976671

For Jupyter, i'm not that familiar with it, but maybe we could generate some sample code to paste into a notebook, based on:

1. https://stackoverflow.com/questions/24853632/how-to-load-a-csv-into-ipython-notebook
2. https://stackoverflow.com/questions/39445623/how-to-import-data-in-jupiter-notebook-from-an-online-csv-file-which-is-in-zip-f

[IshaGupta18]

Oh I think this would be lovely, let me try a couple of these out and I'll get back to you with what worked and what didn't. Thanks a lot!

[IshaGupta18]

So I think a lot of these options are pretty great but I am not able to find a proper or a direct way of implementing these. Is there any way we can speed up this process? @jywarren could you suggest something here or a super simple option that can get the ball rolling? Thanks a lot!

[jywarren]

Hi @IshaGupta18 can you give a brief update on what they mentioned in https://github.com/concord-consortium/codap/pull/144#issuecomment-499232827 and if that'd work?

[IshaGupta18]

@jywarren I didn't get any other reply, apart from the comment that mentioned you. The URL they gave for trying the feature out (https://codap.concord.org/releases/build_0481/static/dg/en/cert/index.html?url=https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD) , if we replace the URL of the file here ie https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD part with the link of another remote file, this doesn't work.

What do you suggest should be done here?

[jywarren]

Can you reply to them showing this and asking if there is a bug? Thanks!

On Thu, Jun 20, 2019 at 12:57 AM Isha Gupta notifications@github.com wrote:

@jywarren https://github.com/jywarren I didn't get any other reply, apart from the comment that mentioned you. The URL they gave for trying the feature out ( https://codap.concord.org/releases/build_0481/static/dg/en/cert/index.html?url=https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD) , if we replace the URL of the file here ie https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD part with the link of another remote file, this doesn't work. [image: image] https://user-images.githubusercontent.com/40794215/59819534-e70e2080-9345-11e9-8005-5e931e83abd1.png

What do you suggest should be done here?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/simple-data-grapher/issues/17?email_source=notifications&email_token=AAAF6J24NCP32GAXQ2NVFT3P3MEZZA5CNFSM4HL3JSQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYEG5TA#issuecomment-503869132, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6J4BQ4W44ARRIG3VZCLP3MEZZANCNFSM4HL3JSQQ .

[IshaGupta18]

They did say before that currently there is a bug in this one, but I will show them this and ask for the solution. Thank you. Also, in the meanwhile, would you recommend some other export option's implementation?