muni-town / weird

Weird web pages
https://weird.one
Other
51 stars 11 forks source link

Chat Importer #96

Open erlend-sh opened 4 months ago

erlend-sh commented 4 months ago

We wanna archive a chat server's complete history, starting with Discord.

This work will happen on two different ends, namely the Iroh storage backend and the Weird client UX.

Iroh

https://github.com/Tyrrrz/DiscordChatExporter https://github.com/slatinsky/DiscordChatExporter-frontend https://github.com/slatinsky/DiscordChatExporter-incrementalBackup

There’s also a Rust equivalent to DiscordChatExporter that might be easier to operationalize: https://github.com/Sciencentistguy/discord-channel-archiver

The deliverable here is quite basic:

From @zicklag:

The easiest way to start off using Iroh for this projcect is, I guess, just as a distributed webserver. Eventually, maybe it could be expanded to be a way to link Discord archives into our wider Fediverse. But first things first! So, Iroh data is stored in what is called "documents" which are a key-value store. The key's can work like filesystem paths by using slashes in the name, for example: /docs/index.html The idea is that we want to create an Iroh Node create a document: node.docs().create() and then take our HTML/JSON export from the Discord exporter tool, and basically document.set_bytes(file_path, file_bytes)

That's not exact code, but to give you the idea.

Weird

For the client-side, this should be really useful: https://github.com/skyra-project/discord-components https://github.com/skyra-project/discord-components-implementations https://github.com/fowled/neptune

Might also come in handy: https://github.com/itzTheMeow/mock-discord-messages https://github.com/domitriusclark/discord-forum-to-website

Later still: https://github.com/SpikeHD/Dorion https://github.com/Vexcited/SolidCord