Sync to non-commercial storage

[PeterJacob]

Cool project guys!

It would be nice to have a sync option to a non-commercial storage, like a self hosted FTP/GIT server.

[Proxiweb]

Or Owncloud...

[explorigin]

Look at vole.cc. They have a custom Go-based server that saves stuff to files. You could probably just plug in into this for easy local storage.

[lunarok]

Yes plug onto an owncloud will be perfect

[Magissia]

Syncing to OwnCloud would be interesting.

[sa3m]

A Bittorrent Sync option would also be nice.

[GustavePate]

+1 for owncloud

[almereyda]

As @michielbdejong wrote about StackEdit, inside there one might find interesting approaches to a generalized multi-provider data storage connector.

Also, remoteStorage is now implemented, too.

[yvangodard]

+1 for owncloud !

[claell]

+1

[Putdeksel]

+1 Cozy.io, Owncloud, Bittorrent Sync. Any one of those would be great!

[andtheWings]

I think this thread demonstrates the need for a standardized method that connects Laverna to storage providers. Everyone has their own preferred program for hosting personal data. There's nothing wrong with that, but now that the developers have provided viable commercial (Dropbox) and non-commercial (RemoteStorage) means of doing so, I would prefer they focus on expanding other aspects of the project. Giving users the tools to connect on their own would free up time in the long run.

[pihentagy]

I would also like Google Drive syncing (or even Google Keep sync)

[SandeepPinge]

+1 - I also agree with @andtheWings.

But for now I think adding 2 services: GoogleDrive (primary, since most people are already using it) and OwnCloud (extensively used Dropbox non-commercial alternative) would cover a very large user base and open Laverna notes to many more people.

This can really expand the community.

[almereyda]

Please note remoteStorage brings Google Drive and Dropbox sync out-of-the-box. Check at 5apps.com.

---

Edit: Bittorrent Sync in the browser sounds adventurous.

[pihentagy]

@almereyda Err, could you be more specific how can I store my notes through remoteData to google drive with 5apps.com?

[SandeepPinge]

@almereyda If by 'out-of-the-box syncing', you mean in your own control, then so does OwnCloud.

But my point is to spread Laverna usage and make it easily adopted by the general public. I believe Google Drive is the missing link there. Most people have a Google account and they can directly use their Google Drive vs restarting with another service signup and setup.

Since most people here prefer OwnCloud (including me) and it is open/non-commercial, I would definitely recommend an option for that being developed first over Google Drive.

[Putdeksel]

I chose to use Laverna, hoping it might evolve into something able to replace Evernone. I'm using OwnCloud to replace Dropbox, Google Drive ect. I took these steps so that I would no longer have to depend on those services and agree with their terms. I get that adding the option to sync with services like Dropbox would add to the popularity (and I encourage that) , but for me personally it doesn't fit.

[GustavePate]

@SandeepPinge

Hurray !

GDrive for the masses / Owncloud for the real ones ! imho, Owncloud / GDrive makes an excellent free / proprietary software combo.

[ThiloL]

I'm running Laverna on my own webspace. Why I can't save the data (encrypted) directly to the webserver? Or can I implement RemoteStorage on my webserver? I haven't SSH access!

[claell]

I think that Laverna was not thought to be a hosted application, but runs local with locally storing the files. I would prefer a hosted solution as well, since it makes synchronisation between devices much easier.

[ThiloL]

Local? If I want local notes I use notepad.exe. ;-)

[andtheWings]

Laverna was designed to be unhosted, i.e. for the data to be stored locally on the browser by default. More info here: https://unhosted.org/

[claell]

"None of us can get access to your personal data because we are using IndexedDB and localStorage. In fact all your information will be stored only on client-side." This means, afaik that the data is stored locally (in a folder that can be accessed by the browser). But one can use Dropbox (and hopefully other alternatives soon) to sync the notes.

[almereyda]

Everyone, please note that remoteStorage is now available in Cozy Cloud, so you can easily spin your own instance.

Additionally, as Laverna supports remoteStorage, but it seems an old spec, already, there is work going on to integrate it into ownCloud, too.

@michielbdejong @skddc Could you have a quick look at Laverna and inspect why it fails to sync with 5apps? It used to work, I believe before the 0.10 release. Did they just not update the client? Would it make sense to submodule it?

[raucao]

I'm not familiar with Marionette or this app's source code. However, we're always here to help and answer and all questions, both general as well as for concrete code.

Whoever maintains RS support in this app (or maintains the app in general): we're very much looking forward to welcoming you in #remotestorage on Freenode or chat about anything on the forums or GitHub.

[romu70]

I would add Syncthing (https://syncthing.net/) and Seafile (http://seafile.com/en/home/) as well.

[iAdanos]

1+ for OwnCloud!

[reimundklain]

1+ OwnCloud

[kalakrumanov]

Owncloud would be great.

[rockneverdies55]

Owncloud +1

[enekux]

owncloud +1

[rhoeberg]

owncloud +1 I'm in excactly the same situation as @Putdeksel Being completely in control of your notes by using your own storage system would be amazing.

[raucao]

Being completely in control of your notes by using your own storage system would be amazing.

Yes, it is amazing and it's already possible via Laverna's RemoteStorage support. You're not even tied to a single server implementation, but any server speaking the RS protocol can store your notes. ;)

[rhoeberg]

@skddc I haven't looked into RemoteStorage to be honest. But sadly from what i can see it doesn't seem to be supported on the server i have from my provider. Hopefully it will in the nearest future!

[frederiiiic]

+1 ownCloud.

[asalthobaity]

OwnCloud sync would be great.

[nikolas-n]

+1 for owncloud!

[raucao]

May I suggest something to all the ownCloud users in here:

It would be much easier for app developers (not just of this app) to support ownCloud, if ownCloud were to support an open protocol for per-user data storage. This app already supports remoteStorage, so if ownCloud were to support remoteStorage, instead of developers having to add special ownCloud support to their apps, then all remoteStorage-enabled apps would automatically also work with ownCloud acting as the remote storage server.

Hence, I think it'd be great if you would ask/vote for, or contribute RS support to ownCloud itself!

[almereyda]

Following this conversation for a few years already, I have to second @skddc 's latest remark.

On 25 July 2016 at 14:22, Sebastian Kippe notifications@github.com wrote:

May I suggest something to all the ownCloud users in here:

It would be much easier for app developers (not just of this app) to support ownCloud, if ownCloud were to support an open protocol for per-user data storage. This app already supports remoteStorage, so if ownCloud were to support remoteStorage, instead of developers having to add special ownCloud support to their apps, then all remoteStorage-enabled apps would automatically also work with ownCloud acting as the remote storage server.

Hence, I think it'd be great if you would ask/vote for, or contribute RS support to ownCloud itself!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Laverna/laverna/issues/6#issuecomment-234938265, or mute the thread https://github.com/notifications/unsubscribe-auth/ABka_MqpAc9pic432ehmtZ58HBUTh2Iyks5qZKqOgaJpZM4BqQ_m .

[Palingenae]

I +1 this issue!

[BentHaase]

Hmm would it be hard to implement webDAV as this could be seen as an open standard which will allow laverna to be connected to dozens of different storage providers?

[wangzk]

Why not store the data in a specified local directory and people can sync their data into cloud via sync clients like Seafile or ownCloud Desktop clients?

[raucao]

Why not store the data in a specified local directory and people can sync their data into cloud via sync clients like Seafile or ownCloud Desktop clients?

That's only possible in a potential packaged version of the app, but not on the Web. It's also what remoteStorage solves and why it's used in Laverna.

Did anyone here open an ownCloud issue about integrating remoteStorage so far? I can only stress how great that would be and that Laverna would then automatically support ownCloud.

[Palingenae]

That's only possible in a potential packaged version of the app, but not on the Web. It's also what remoteStorage solves and why it's used in Laverna.

I actually never succeeded to have a fully functional remoteStorage synchronization . I tried many times, and it's rather hard to deal with everybody everytime. I am not so comfortable with IndexedDB datas...

I am rather familiar with the DAVs syncs. @wwebfor told me there's another way to sync in the oven, thought.

[raucao]

I actually never succeeded to have a fully functional remoteStorage synchronization . I tried many times, and it's rather hard to deal with everybody everytime.

Are you saying it failed after the sync fixes a while back, and you opened an issue about it, that didn't come to a conclusion?

I haven't seen any such issues or requests, but I'd be happy to help you! Not sure what "hard to deal with everybody everytime" means.

I am not so comfortable with IndexedDB datas

Well, it's basically the only database available locally in modern browsers (except for localStorage of course), so there aren't really options for using something else.

[Palingenae]

Are you saying it failed after the sync fixes a while back, and you opened an issue about it, that didn't come to a conclusion?

No, I asked on Gitter, and I came to the conclusion that I must clean all my data on every browser I used as it appears that it properly works on clean browser (where Laverna have never been used).

Not sure what "hard to deal with everybody everytime" means.

Each browser (or desktop client) I use, each time i use one of them.

Well, it's basically the only database available locally in modern browsers (except for localStorage of course), so there aren't really options for using something else.

Yes, yes, I understand. But I meant that I am not used to this way of syncing datas.

[raucao]

Sounds like some upgrade issue to me. So, when you write...

I actually never succeeded to have a fully functional remoteStorage synchronization

... does that mean it still doesn't work? If that's the case, then I'd still be happy to help.

[popindavibe]

+1 for webDAV, much more standard than an "Owncloud only" solution. Let's stay Open guys.

[technodrome]

Instead of integrating a whole lot of nonsensical new hipster solutions like remoteStorage, why not use tried-and-tested webDav? I mean what's the point? Why are you forcing users to bend to your will if industry standard is de facto something else?

Saying yeah, rs is supporting this and that is of little use to most of the folks. But I do not want to use yet another protocol I do not care about. You support commercial Dropbox, but OWNING the notes means I do not want to save them to a 3rd party server. You just do not want to understand your software will survive only if people want to use it because its friendliness and flexibility. Instead, you aim to strong-arm them into something they don't want to do. These are all devs and tech-savvy people, you just don't get this simple fact.

[raucao]

@technodrome I'm not speaking for the Laverna devs, but as a remoteStorage.js developer. Maybe it's not clear, that my trying to help people here (I never tried anything else, if you carefully read the history) is not the same as telling people to go away, because their opinions aren't valid or heard by someone else. This is certainly the right place to voice them as far as I can see, and I'm not sure why your first comment in this thread is so hostile. Let's all be nice to each other, because this is unpaid voluntary work for all of us!

Instead, you aim to strong-arm them into something they don't want to do

I don't see how providing an open-source software, free of charge, is strong-arming anyone. If these are all devs and tech-savvy people, surely someone can contribute WebDAV support, if it makes sense?

So, let's look at the viability of WebDAV objectively then. Here is a (probably incomplete) list of the problems I can think of, that would need to be solved for WebDAV to be integrated. (These are basically the very reasons why the remoteStorage protocol was created in the first place. In fact, RS was using WebDAV in the very beginning, and the authors decided to remove it in favor of a simpler REST API, based on real-world testing and experience with that):

1. CORS

The way browsers work is that an unhosted app like Laverna connects directly to the server from JavaScript. This carries with it the restriction, that the server must offer Cross Origin Resource headers for all requests and support OPTIONS pre-flight requests for HTTP verbs that can manipulate data, like e.g. POST, PUT, DELETE. Most WebDAV servers do not support this. So a lot of WebDAV servers would be incompatible with Laverna out of the box.

Problem: even if one would not care about those servers being incompatible, how would an app detect if support is there? This is actually made much more difficult by browsers intentionally not giving JS code details about why the connection failed, so there would need to be some kind of discovery mechanism. Also, it has to be explained to the user what exactly could be the cause and that they might have to switch servers in order to use the app.

Solution in RS: the protocol requires CORS out of the box. Also, discovery is handled via Webfinger, where the user address can return all configuration info, but also where extra capabilities of the server can be announced.

Solution for plain WebDAV/Laverna: Any ideas?

2. Permissions / Authorization

Permissions have to be handled server-side and per account in WebDAV, as there's no such thing built into the protocol. Authorization of certain folders to access doesn't exist.

Solution in RS: OAuth 2.0, so app developers can decide what part of the storage they need access to, and users can easily see which part that is, and decide to grant access in a simple dialog.

Solution for plain WebDAV/Laverna: This is basically impossible, but it would of course be possible to just skip that feature and voluntarily give Laverna access to all of one's data. Not elegant, but certainly not a showstopper for implementation.

3. Offline/mobile support

This is not particularly a problem of WebDAV or solved in the RS specification alone entirely. But for a mobile-capable web app, it is nonetheless an important factor. Both desktop and mobile connections drop regularly, and one wouldn't want their notes to not be saved because of it. So some kind of offline storage needs to be used, and intelligently synced with the remote server (and of course without re-auth after every drop if you want to create a good user experience).

There are two problems here:

Problem 1: Servers need to support Etags, so one is able to build a sync mechanism in the first place

Solution in RS: This is partly solved in the protocol by requiring ETags on both directory resources and their item listings, as well as single documents themselves. Based on that, one can build sync mechanisms using HTTP headers like e.g. If-None-Match and responses like e.g. empty 304s for requesting things that haven't changed and 412 for refusing to overwrite something that has changed on a different device at a different time.

Solution for plain WebDAV/Laverna: Unfortunately, Etag support is only a SHOULD in the WebDAV spec, so many servers don't support it, and some even ask their users explicitly to turn off support, even if they have it, because it's implemented in a non-performant way. (This is actually not an easy problem for a server developer; I can certainly attest to that). But similar to CORS, it is possible, so the solution here would again be down to feature discovery and guiding users to configure their server or tell them they have to switch to another server. Fortunately, this is much easier than with CORS, because one can detect this from JS by inspecting response headers.

Problem 2: Write an actual sync mechanism or use a library that provides one

Solution in RS: Very early on, one of the spec authors started creating a reference client library to solve this issue for people who want to integrate RS into their apps. It is still being maintained and developed and has been battle-tested on hundreds of thousand of different connections, devices, OSs, browsers, in Cordova, and so on. It is also being used in Laverna. The first commit of that goes back to Nov 2010, so that certainly isn't a "new hipster solution". If there are bugs with it, there are people who are glad to help with those, and any feedback was and is always appreciated (hence my proactive responses in this thread).

Solution for plain WebDAV/Laverna: One possible solution would be a custom hand-rolled library. The effort to create, fix and maintain such code would certainly dwarf the other problems I listed, and yet it's of course very possible to do it. The main challenge I would assume, apart from literally a thousand other things to implement, is probably variable behavior of WebDAV servers in the end, esp. their Etag implementation. However, if you know of such a library (I know of none, and I also couldn't find one, when I checked again just now), it would become very possible to solve the other outstanding issues in some way and provide WebDAV to technical users.

---

Please excuse and point out any mistakes or wrong facts contained in this list. I basically wrote all of it off the top of my head. I'll transfer this to the RS wiki sometime soon, and ask some people to review/edit what I wrote, as I think that this is actually a rather useful explanation of the differences between WebDAV and RS, and how RS solves things that WebDAV hasn't, when it comes to client-side Web apps that run entirely in the browser (which is understandable, as the Web platform capabilities didn't really exist back then and the whole concept of offline-first web apps wasn't a thing).

@technodrome If you want to progress your goal of getting WebDAV support, then pointing out potential solutions to any of these problems (or facts that void them entirely) would probably go a long way.