Solid on non-pod backends

[jeff-zucker]

Following some discussion with @justinwb , we decided that non-pod interoperability isn't the main focus of this panel but that it's a related issue and belongs here. I'm glad to hear other suggestions but will start here for now.

I have begun pulling together use cases related to using Solid on non-pod backends (file systems, in-browser storage, non-pod cloud storage,databases). If anyone knows of some existing work or has suggestions, please get in touch.

[happybeing]

Thanks for doing this Jeff. I think my work putting Solid on SAFE as a proof of concept probably fits here. This is a reasonable summary of my work:

https://safenetforum.org/t/solid-on-safe-updates/29318?u=happybeing

I'd also highlight this post of mine on the synergies which have lead me to advocate for integrating Solid and SAFE:

• Why Solid on SAFE?

Not specifically Solid but also relevant and significant has been Maidsafe's addition of LinkedData and related technologies, and their own proof of concept demonstrated at the dWeb Summit where they met TimBL (and possibly RubenV). See Maidsafe at Decentralised Summit (2018)

[justinwb]

@jeff-zucker when you talk about non-pod backends, do you mean data is stored in a location that would not be accessible through the Solid API, but would be manipulated externally and then (likely) synchronized back to a pod through the Solid API (and vice versa)?

[jeff-zucker]

@justinwb - The short answer is that one can either use (most of) the Solid API directly on the backends or as a way to transfer between the backends and pods. The longer answer below the pic.

Solid-Rest provides most of the Solid API (minus enforcing access control and a few other things) for a variety of backends.  So far it works on local file systems, on in-browser storage such as indexedDB and Native File API, and on Dropbox and other backends supported by the BrowserFS project.  And Solid-Rest has a (not well documented but existent) API for plugins so other backends such as local databases can be leveraged in eventually.

This means that Solid tools (rdflib, etc) can be used on  data on these backends without installing either an http or solid server.  One use of this is for transferring files between these backends and pods using the same fools on all the platforms and using linked data navigation rather than the physical file structure regardless of whether on the pod or not.  Several existing apps by @alvetens use this feature for deployment - the user npm installs the app locally, then runs a script which deploys the app's container tree on the users pod.  Solid Gitter uses it as a way to store chats locally. A number of people use it to backup their pods. The Data Kitchen uses these features to run the mashlib browser against any of the backends and pods at the same time.  This opens the possibility of local semantic desktop as well as using it as a way to test things locally before uploading to a pod.  It also supports using local apps (e.g. emacs or protege) against both local RDF and pod data within the same environment.  So essentially it means that the same toolsets and reasoning can be used against local, cloud, and pod storage without switching apps. Another use for it is testing - Solid File Client has a very extensive test system which can be run interchangeably against a pod or the local file-system, or an entirely in-memory storage. Another use is configuration files - an app can store its local configuration in a turtle file and and use follow-your-nose to find it and solid-ui to parse and display it.

I'm trying to rephrase all of that as requirements and use cases in the format of the use case doc you pointed me to. It strikes me that this all may fit under problem #4 in the interoperability problems and goals: #4 "People shouldn’t need to think about how to physically organize their data to use Solid." ... #4a "Nor should they have to think too much about where they are storing it and should be free of the physical organization in that storage as well."

    

[csarven]

Related https://github.com/solid/specification/issues/119

[jeff-zucker]

Also related : Solid-Rest currently works by altering the fetch in solid-auth-cli and solid-auth-client, this will be changing since both those packages are being replaced by solid-auth-fetcher, See https://github.com/solid/authentication-panel/issues/45.

[jeff-zucker]

I could be wrong but https://github.com/solid/specification/issues/119 seems to relate to what happens on the server - i.e. how the server should handle things like index.html which may have been created manually. Solid Rest is entirely a client-side interaction and by the time the server sees anything, it should be seeing the same output from a Solid Rest retrieval of a file on the local system as it does from a solid-auth-client retrieval of a file via https including a full Response object with expected Solid headers. So, support of Solid Rest would not (and should not) demand anything different in the server spec.

The question is, perhaps, should there be a spec for "Solid lite" (sorry, yuck) which defines a minimal "Solid like" API which operationally I have been defining as an API that supports the mashlib databrowser.

[jeff-zucker]

As a historical note, I have been pursuing this entire area with Tim's encouragement. He seems to agree that file:// and other spaces are part of the Solidverse. See his message in the forum : https://forum.solidproject.org/t/support-for-file-need-help-testing/1683/2.