Channel logs and history/archives

[raucao]

Tracking/discussion issue for all things channel logs.

[raucao]

Public logging to RS already working:

• https://github.com/remotestorage/modules/blob/master/src/chat-messages.js
• https://waves.kosmos.org/

Can be used to diff logs between last own logs and coming online again (only in channels where someone or something logs to RS in that format, of course).

[raucao]

The super low tech first MVP would be to just always show the logs from Kosmos's public storage in supported channels when opening the app, and not even log to everyone's storage at first. That way, we can demo the app and functionality without implementing the perfect solution first.

[raucao]

The super low tech first MVP would be to just always show the logs from Kosmos's public storage in supported channels when opening the app, and not even log to everyone's storage at first. That way, we can demo the app and functionality without implementing the perfect solution first.

This has been implemented yesterday by @galfert and is already in master. 🎉

It finally makes it possible to actually use Hyperchannel without the need for Waves or other additional tools. But it's also rather clunky as it is now. One major problem is that one has to reload the whole app in order to get logs since having used it last (usually just a couple hours in between and with the app open in some tab or an add-to-home app. So we really need to make this a bit more sophisticated asap.

The easiest next step is to reload that archive file regularly or at least when getting page visibility of course. But then we'd probably want to write messages to the unconnected RS for now, and only add the new ones when fetching updates. Also, we should use If-None-Match for the refetching, of course. That way, we can get empty 304s when there aren't any new messages.

Main problem with IRC is still the identification of messages, as they don't carry IDs and timestamps are unique to the device. My initial idea was to give them a composite ID, created from the hashed message body and a timestamp in seconds (because the difference in time between hubot and sockethub is usually milliseconds). I guess we'll have to play around and test this with actual usage in order to find an acceptable solution.

[raucao]

The content in this issue is a bit ouf date, and doesn't consider our default choice of XMPP as messaging protocol. As XMPP messages carry IDs, we do not have the same issue with it.

Also, message IDs are a draft spec for IRC v3. So I think we should just wait for them to land natively, and live with non-existing message archive sync for IRC for now.

Choosing a device for archive sync, or don't sync archives to RS at all

However, if you're connected with more than 1 device, we have to think about how to handle the archive sync, without multiple clients trying to write archives at the same time, because it would result in sync conflicts all over the place. For this, we can use the normal presence notifications.

I was thinking about adding some more details to our device name, e.g. if it's a mobile device or not (based on screen size). If you're online with both desktop and mobile at the same time for example, we could then select the desktop device over the mobile one to sync archives, because we can assume that the connection is more stable (and sometimes also faster) on it.

[silverbucket]

In order to solve the constant writing to RS and resolving conflicts for multiple apps writing the same message, we could instead have a periodic task that will fetch the recent XMPP archive logs and then save them to remoteStorage.

[raucao]

Just a note to clarify: most of the conversation here is about history/archive sync, and not purely about client-side channel logs. However, before we sync history we need to have local history in the first place. I just created #237 for that.