CaptainIRS / programmer-lyf-bot

Your Daily Dose of Programming Content via IM (ft. GitHub Actions)
https://t.me/ProgrammerLyf
MIT License
9 stars 1 forks source link

Add channel for comic strips #12

Open CaptainIRS opened 3 years ago

CaptainIRS commented 3 years ago
vaishnavi192 commented 5 months ago

Hey @CaptainIRS I would like to work on this issue under DWOC'24 but can you plz tell a flow how to do this. We have to store data in json format, web scraping will also be used?

CaptainIRS commented 5 months ago

Hi @vaishnavi192,

The ideal flow would be: (mostly based on the logic behind how forum posts are fetched)

  1. Collect a list of comics with content related to programming/programmers/software., like XKCD, MonkeyUser, The Developer's Life, etc.
  2. Create a config file for comics in the config, folder, similar to config/forums.json, which defines the frequency of posts, name and icon of the site, etc.
  3. Create a comics/ folder in the root similar to the forums/ folder to create files to fetch the comics from the sites and process them. (Refer next step)
  4. Check if the comic has an RSS feed. To do that follow step 3 in CONTRIBUTING.md#adding-new-feeds. If there is a feed, then write the logic like forums/slashdot.py, else create something like forums/hackernews.py to scrape the site yourself and get the posts according to the frequency.
  5. Create a comics/__init__.py like forums/__init__.py to act as an interface.
  6. Create utility functions in util/embed_utils.py similar to create_forum_embed, create_forum_telegram_post and create_forum_reddit_post to convert your post to the formats recognised by Discord, Telegram and Reddit.
  7. Add a new channel for comics in main.py (here) and update .env.example accordingly.
  8. Create an updater in main.py like _forum_updater and add that to the logic in DiscordBot, send_telegram_messages and run_reddit_bot.

It might seem like there are a lot of steps, but each step is really only a few lines of code and I've tried to map out all the steps exhaustively. Do let me know if you have any questions...