A website where users can setup ETL with a custom service so they can use that data in Solid without having a technical background.
Desired solution
In order to enable Solid to scale to interoperate with a large number of user services; we need a way for non-techincal users to be able to easily define an ETL pipeline between an existing service that they use, and their Solid POD. The way that I see this working long term is that:
There is a "Link my Pod" app designed specifically to allow me to link my Pod to existing services (there is a discussion to be had here about whether each service should be a Pod of its own that we federate over, but I'll ignore that technical detail for now).
Within the app I can search through existing applications that I can link with based on what the service already has ETL pipelines defined for (for instance I can look up Spotify within the application, select Spotify - and then be directed through the Spotify auth flows). Once I have completed the auth flow I should have a new agent/orchestrator in my Pod that will use and maintain those credentials in order to maintain an up-to-date reflection of my Spotify data within my Pod.
If I want to import a more niche application that there are currently no pre-defined RML mappings for; there should be an user interface to be able to define these mappings. Once I have finished defining the mappings, they should be saved to my Pod - and the agent/orchestrator for my Pod is set-up. At the same time my Pod should send a LDN to the RML mappings Pod to notify it of this new, public, rule-set that I have created; which other users can make use of (this flow should eventually be extended to include notions of other users rating, verifying mappings - and strategies to handle multiple rule-sets being published for the same target data source. But I would consider this an out-of-scope future problem for now).
Pitch
A website where users can setup ETL with a custom service so they can use that data in Solid without having a technical background.
Desired solution
In order to enable Solid to scale to interoperate with a large number of user services; we need a way for non-techincal users to be able to easily define an ETL pipeline between an existing service that they use, and their Solid POD. The way that I see this working long term is that:
Acceptance criteria
An application which I am able to follow the above flow with Spotify and google calendar (to relate this to https://github.com/SolidLabResearch/Challenges/issues/68).
Pointers