Open CaptainIRS opened 3 years ago
Hey @CaptainIRS I would like to work on this issue under DWOC'24 but can you plz tell a flow how to do this. We have to store data in json format, web scraping will also be used?
Hi @vaishnavi192,
The ideal flow would be: (mostly based on the logic behind how forum posts are fetched)
config
, folder, similar to config/forums.json
, which defines the frequency of posts, name and icon of the site, etc.comics/
folder in the root similar to the forums/
folder to create files to fetch the comics from the sites and process them. (Refer next step)forums/slashdot.py
, else create something like forums/hackernews.py
to scrape the site yourself and get the posts according to the frequency.comics/__init__.py
like forums/__init__.py
to act as an interface.util/embed_utils.py
similar to create_forum_embed, create_forum_telegram_post
and create_forum_reddit_post
to convert your post to the formats recognised by Discord, Telegram and Reddit.main.py
(here) and update .env.example
accordingly.main.py
like _forum_updater
and add that to the logic in DiscordBot
, send_telegram_messages
and run_reddit_bot
.It might seem like there are a lot of steps, but each step is really only a few lines of code and I've tried to map out all the steps exhaustively. Do let me know if you have any questions...