Protocol versions in README

[raucao]

Hi,

I just wanted to check if reStore supports a protocol version newer than -01 (<= -02 is deprecated from Dec 15 on), and stumbled upon this part of the README:

reStore remoteStorage server written for Node.js. It is designed to be compatible with the 0.6 (RemoteStorage-2011.10) and 0.7 (RemoteStorage-2012.04, draft-dejong-00) versions of remoteStorage.js.

Looks like that parts is outdated, but I'm not sure which versions it supports or doesn't support at the moment, which is why I'm opening an issue instead of a pull request.

Cheers, Basti

[jcoglan]

Thanks for alerting me to this; it's fixed in https://github.com/jcoglan/restore/commit/4a7be869b4f65ff9a26d4a8537aa5f0f90f863a7.

The intent is to keep up compatibility with RemoteStorage.js (I have added examples using 0.10 and 0.11 now), while maintaining backward compatibility where possible.

[raucao]

Ok, great!

Allow me to clarify the whole thing a bit, because I think there are some misunderstandings here.

remoteStorage.js is in fact keeping compatibility with the remoteStorage protocol. Servers can only be compatible with a specific version of the protocol, which in turn is supported in remoteStorage.js for at least the last 3 protocol versions. The version you announce in Webfinger records is the one the server needs to support all required features of.

As protocol changes are rarely breaking, but in the case of directory listings actually were (between -01 and -02 I believe), it's logically impossible for servers to support older remoteStorage.js client versions which don't yet support the newer protocol versions. In this case it means if your server conforms to the -02 protocol, it is automatically incompatible with remoteStorage.js <=0.8.

Protocol versioning

and draft-dejong of the protocol

Versioning switched to using the draft-dejong prefix to make it a couple of years ago. The GitHub repo has some info about the current scheme. TL;DR: currently supported are draft-dejong-02 to -04, with the newest version always ending recommended support for the oldest one every 6 months. As things are fairly stable now, this usually doesn't break much anymore and is rather easy to track for server implementors, but as I said, it did so for -02 at least.

The Webfinger format for the protocol version also changed to make it easier to find the protocol by using full URIs for the value.

We agreed to start bringing the protocol on RFC track later this year, so I guess we might actually do that in 2016 sometime and then have it be "finished" until a major new version with new features makes sense (probably for collab/ACLs or similar stuff).

And now?

For the README, I would propose describing your intent to stay compatible with new versions of the protocol, as well as stating the exact version it is currently using.

Let me know if you have any questions about all of this!

[raucao]

Friendly reminder that this should be re-opened and updated sometime.

[raucao]

@jcoglan I could prepare a pull request for this, if you don't have time for it. I would've done it already, but I'm slighly uncomfortable describing your intentions for you. Maybe you can just have a quick look and rewrite that one sentence/paragraph sometime?

[raucao]

@jcoglan Don't want to be nagging. Could you re-open and either help me prepare a PR or put it somewhere on your list maybe?

[jcoglan]

@skddc Thanks a lot for explaining the intention. Is it problematic for a server to support multiple versions? I realise this isn't always possible due to the nature of some of the changes, but I'd like to provide broad compatibility if I can, in places where it is possible.

For example, one could support both the root:rw and *:rw scope syntax at the same time. One could run dejong-01 while incorporating some features (e.g. response headers) from other versions.

I'm just thinking that there will be clients other than the official one (I already have one myself), and I like backwards compatibility in situations where it doesn't cause undue problems. Which reminds me: is there a guide for using the official client in Node? I know this was supported but I can't figure out how, and have ended up doing some of my own tools for particular use cases I have.

To clarify my thoughts about this I've taken down some notes on how the spec has changed over time so I have a clear picture of what's required in each version. This should help make sure reStore has a consistent set of semantics.

[jcoglan]

As a side note, one thing strongly putting me off updating to dejong-02 or later is the breaking change to the folder description response format (though a client could detect this), and the requirement that items be able to use any char and that folders and documents not have conflicting names. I believe the spec at one point explicit said you could have name collisions, and I implemented as such, but having gone back over the spec documents I can't find a reference for this.

These changes would mean the naming scheme for the file system backend would need to change again, and I'm not sure yet if I can automatically migrate people's data safely. But most likely, the next release will stick with 01 and we can figure out later versions at another time. There's already too much backed up unreleased work so I should fix that asap.

[silverbucket]

Hi @jcoglan good to see you back again, as for running remoteStorage.js from node.js, there's a brief doc here: https://github.com/remotestorage/remotestorage.js/blob/master/doc/nodejs.md

[raucao]

Is it problematic for a server to support multiple versions?

For example, one could support both the root:rw and *:rw scope syntax at the same time. One could run dejong-01 while incorporating some features (e.g. response headers) from other versions

Yes, it's certainly possible with some features, yet impossible with others. You can easily support multiple permission scope formats, like you said, but you can only implement one directory listing format, of course.

In general, there's no concept (yet) for negotiating spec versions between client and server. That means you can only decide on one protocol version to announce and support, with compatibility then being dependent on how compatible a newer spec version is with an older one.

As stated before, the latest versions don't contain any (critical) breaking changes anymore, so getting reStore beyond -02 seems to be the biggest hurdle.

one thing strongly putting me off updating to dejong-02 or later is the breaking change to the folder description response format (though a client could detect this)

Yes, this one should be rather simple. I upgraded some apps myself and it was usually quick and painless.

And the requirement that items be able to use any char

Seems to be possible. At least you don't have to rename old ones for this.

and that folders and documents not have conflicting names.

This seems like the difficult one to me, as you might have old items conflicting with this and the question obviously is how to handle that.

It was introduced due to the fact that sooner or later people will want to mount remoteStorage accounts as filesystem(-like) volumes and on most systems that means you cannot have a directory and file have the same name. So trying to get it removed again from the spec is a hard sell I think, removing that option as a solution for us here.