keplergl / kepler.gl

Kepler.gl is a powerful open source geospatial analysis tool for large-scale data sets.
http://kepler.gl
MIT License
10.38k stars 1.74k forks source link

Backend Storage (I): Persisting MAPS (privately) #805

Open ana-md opened 5 years ago

ana-md commented 5 years ago

1. Basics

Is your feature request related to a problem? Please describe. Kepler.gl has a 'Save/Share your map' feature using Dropbox but it doesn't have a way to manage privately the maps after its creation. CARTO is already working together with Kepler in the implementation of the option for sharing maps via URL using CARTO, but there is still no way to allow the user to save the map privately.

Describe the solution you'd like An integrated and private system to easily persist, load and manage maps from an external provider (to be extended in a next issue for datasets).

Describe alternatives you've considered

2. Mockups & Prototype

After getting some feedback from Kepler directly and evaluating the technical challenges internally, this is our proposal for a Kepler user that wants to privately save, manage and load back maps they have created in Kepler: https://projects.invisionapp.com/share/TPTSVDX2KF5#/screens/389126759

Find screenshots of the main mockups below: Saving Map: Storage-menu save-map-modal

Loading Map: Load

Managing Maps from CARTO: Carto-dashboard

3. Technical Notes

3.1. Generic approach

We propose a generic approach, in which any cloud provider with an OAuth API can include a predefined set of methods to implement the backend storage feature. And that will allow the user to persist maps (and datasets), with privacy control, and to get them back from the provider with a set of visual components in Kepler (a new section in the "Add Data to Map" modal).

The first method would be one for declaring provider's capabilities. We're proposing these:

a) map public sharing (public url, like dropbox provider currently does) b) map private storing (restricted url)

We propose these main methods:

3. 2. Internal notes about CARTO storage provider

The basic mechanism would be a set of simple HTTP remote calls underneath to use the persistence capabilities of the CARTO platform APIs (namely the SQL API). Those API requests to read & write maps would be encapsulated in a CARTO JavaScript library, so the code should be quite concise in the kepler.gl side. Internally, a set of specific tables would be created in the CARTO user database, to manage public and private maps.

After a proper integration, the user would be able to manage (list, share, delete) the maps previously created from the CARTO Dashboard.

Note: In our particular case, the data for the map will be stored separately, since it maps nicely to the concept of a dataset in CARTO. This is an implementation detail, and other providers could choose to store everything together, eg. on a single JSON file (config + data) and not provide any dataset management capability.. We will tackle this datasets specifics in a next phase.

heshan0131 commented 4 years ago

Great start!

My thoughts are in the current design Save and Share are 2 different actions. But for a user, the 2 are essentially the same. If one wants to save a map to Carto, it should be obvious to them that they also would use Carto to share it. As a result, if one already save their map to Carto, it should be the default selection in Share map panel. Even better, as soon as they click share map, the Carto URL should be generated.

Also, there is currently no way for one to log out of the cloud storage account they selected (dropbox or Carto) and use a different account. Could you think about how that can be done?

ana-md commented 4 years ago

Thanks a lot for your feedback @heshan0131!

Find below my answers to your questions:

  1. Regarding your first point, yes, for the user that should be clear, thanks for pointing it out! We will take a look at it with the team and come back to you with a design proposal.

  2. We included the log out option in the InVision prototype from the Save entry point > Settings and it should appear when you are already connected. Would that be enough?: LogOut

heshan0131 commented 4 years ago

The in screenshot above, would it be possible to display the name of the user that is currently loged in? e.g. Log in as heshan0131

How is versioning handled here? If I saved a map named 'NYC Taxi', when I update it, will the UI make it clear that the map I am going to save will override the old one? Or is it possible to save as a new map?

heshan0131 commented 4 years ago

For the metadata associated with createVisualization could you add a date to it?

ana-md commented 4 years ago

@heshan0131 you can find the latest prototype here, modified including your feedback: https://projects.invisionapp.com/share/TPTSVDX2KF5#/screens/384524982_Sharing_-_Home

Additional comments and screenshots about the changes:

  1. My thoughts are in the current design Save and Share are 2 different actions. But for a user, the 2 are essentially the same. If one wants to save a map to Carto, it should be obvious to them that they also would use Carto to share it. As a result, if one already save their map to Carto, it should be the default selection in Share map panel. Even better, as soon as they click share map, the Carto URL should be generated.

This change has been included in the design, generating the CARTO URL automatically when they click share map and they are already logged in to CARTO. Besides, we have made it clear in all the connection screens that you can only have one active provider at a time.

  1. Also, there is currently no way for one to log out of the cloud storage account they selected (dropbox or Carto) and use a different account. Could you think about how that can be done?

    The in screenshot above, would it be possible to display the name of the user that is currently loged in? e.g. Log in as heshan0131

Log out was added already from the save panel but not from the share and the load panels. Now we have included it. Besides, we have added the name of the user as you proposed: LogOut1 LogOut2

  1. How is versioning handled here? If I saved a map named 'NYC Taxi', when I update it, will the UI make it clear that the map I am going to save will override the old one? Or is it possible to save as a new map?

Versioning was already handled with a "Save as" option in the previous prototype version, but he have now added a screen when you press "Save" that makes clear for the user that he is overwriting the old map. See below: SaveAs Override

  1. For the metadata associated with createVisualization could you add a date to it?

We have added the last modified information to the "Load" panel: LoadDate