Archiving and providing search for discord Q/A channels (especially new Dicsord Forums, used by various software projects).

NathanC commented 1 year ago

Project description

Recently, a few tools or libraries I've used have Discord servers with help channels—sometimes using the new Discord Forum Channels feature. For example, I was learning Blender recently, and used their Discord extensively to get quicker answers than I could on some Stack Exchange. The answers were quite helpful, and I liked the more synchronous nature of conversation. An open source React library I'm using also used the same pattern, although I believe they just used a normal (non-forum) discord channel with threads and some bot to coordinate answers.

However, for the Blender server specifically, my questions were deleted/removed after a few days, disappearing into the ether. I couldn't reference the old questions, nor learn from other people's old questions. And for both servers, I couldn't find any of this info via a search engine.

It seems like a huge waste of expertise! Having a discord bot (or potentially working with Discord?) to archive these forums in some publicly searchable database would be a huge benefit. Open information is incredibly beneficial, and seeing stuff like this disappear into a closed framework is disheartening.

It could be pretty basic to start out with, a discord bot to archive forum posts, a server with a DB to store the data, and a frontend to search/view the data.

Discord servers could control the privacy settings, maybe stripping user's names from the forum posts, and of course notifying users that their questions/answers will be public. If we wanted to have consistent identities across a Q/A channel or even across discord, we could HMAC the user IDs, though that has risks. Maybe user's opting in to having their name's associated with it?

I know using Discord is sometimes frowned upon for open source projects, and for good reason, but it's hard to deny its convenience and I'm sure a lot of projects already use it. We could push for more open platforms like Matrix, while at the same time making the info in tech communities on Discord more searchable.

I've digressed a bit. tl;dr; Q/As in discord for technologies/libraries are not searchable via search engines, and often lost entirely. It would be a great win for open data to archive and make available these resources

Relevant Technology

Discord bots can be written in a variety of languages. I'd personally recommend discord.js + TypeScript.

As for the backend, how we'd archive the data and present it, there's a lot of options and to be honest I don't understand the space particularly well. Maybe using some sort of Stack Exchange and importing the forum posts as questions/answers? Would be hard to structure though. I also recently stumbled across an open source Angolia alternative, MeiliSearch for indexing and fuzzy search.

Frontend to view the data could be anything, I'd personally advocate for React + Typescript (unless that would cause issues with search engine indexing).

Complexity and required time

Complexity

[ ] Beginner - This project requires no or little prior knowledge of the technolog(y|ies) specified to contribute to the project
[x] Intermediate - The user should have some prior knowledge of the technolog(y|ies) to the point where they know how to use it, but not necessarily all the nooks and crannies of the technology
[ ] Advanced - The project requires the user to have a good understanding of all components of the project to contribute

Required time (ETA)

[ ] Little work - A couple of days
[ ] Medium work - A week or two
[x] Much work - The project will take more than a couple of weeks and serious planning is required

open-source-ideas / ideas