Get dumps from Riot - Githubissues

jbelien commented 5 years ago

Following the really nice report Tim did a year ago, @joostschouppe and I talked about creating a daily (weekly ?) text dump of our Riot channel(s).

joostschouppe commented 5 years ago

Some thoughts on this:

we should think about user buy-in about the conversations being archived. They of course are already, but people might be surprised if old conversations become more accessible
people can remove their comments from the stream. I guess once in the dump, they're there forever (we could manually remove I guess). It would make sense to grab the data only after 24 hours or so, so that a message that was instantly deleted doesn't re-appear there
as for publishing format, it would be nice if it's human-readable. So just plain txt or html would be my preference. Or we also provide a human-readable format as well
what kind of dump size would be preferable? Daily/weekly/monthly/yearly/one-huge-file?

hgcvm commented 4 years ago

I find this an interesting project and I'm willing to work on if needed.

I only googled shortly, but came across this tool that might be worth looking into: https://gitlab.gnome.org/thiblahute/matrix-dl

Would there be a server available from osmbe to run this on?

If all else fails, another option is to save dumps from the linked IRC channel. For sure plenty of tools for doing that.

jbelien commented 4 years ago

I find this an interesting project and I'm willing to work on if needed.

Any help is more than welcome \o/

Would there be a server available from osmbe to run this on?

We indeed do have an OSMBE server but for production ready solutions only (mainly because it runs all our production tools) !

There is not much documentation on https://gitlab.gnome.org/thiblahute/matrix-dl ; could you give it a try and show us how it works and show us the end result ?

jbelien commented 7 months ago

Thanks to @thibaultmol, we know have https://github.com/osmbe/matrix-backup 🥳

thibaultmol commented 7 months ago

oh right.... forgot this issue even existed XD

btw: attachments/files (whatever you want to call them) are hard to export because of the way Element implemented chat export atm. Often it just crashes when trying to export them. And doesn't seem like there are other exporting tools atm (that actually work and are still maintained). So I'll get on that, once that is fixed

osmbe / community_planning