playscholar

playscholar

An automated process to summarize a web page URL with ChatGPT.

Bug report · Feature request · Read Docs

Purpose

This document describes the playscholar automated process to summarize a web page URL with ChatGPT.

Audience

The audience for this document includes:

• User who will send a command with a web page URL via a Telegram Bot, and receive a text with summarization of its content.

• DevSecOps Engineer who will design system workflows, configure and manage any SaaS or selfhosted services, and plan for disaster recovery.

Why playscholar?

1. This summarization-as-a-service leverages on and applies a similar design from the playscribe automation to its workflow.

2. This services uses resources in both a free tier and a pay-per-usage tier by cloud SaaS providers:

   Free	Pay-per-usage
   Pipedream	ChatGPT
   GitHub	jina-ai/reader
   Telegram

3. This service is inexpensive, easy to implement, and can be duplicated for other services.

4. This service does not require implementing a custom frontend as the User will interact with a Telegram Bot.

5. This service does not require a backend server as the processing will be performed using ephemeral runners.

6. This service uses a distroless image that drastically reduces the image size, resulting in a faster execution time for its emphemeral runners.

7. Scraping can be complex and often blocked, and raw HTML is cluttered with extraneuous elements like markups and scripts. The Reader API addresses these issues by extracting the core content from a URL and converting it into clean, LLM-friendly text, ensuring high-quality input for your agent and RAG systems.

8. Reader API is multi-modal which means that images on the web page are automatically captioned using a vision language model in the reader and formatted as image alt tags in the output. It natively supports PDF reading and is compatible with most PDFs.

9. Reader API is available for free and offers flexible rate limit and pricing.

Limitation

This project has several limitations:

• The Telegram Bot has a content limit for messages sent to the User.
• The pipeline job may exceed the time of the fixed delayed step in the workflow.
• The message is truncated after 40 lines and will include a link to the file examples/result.txt. However, the content of this file may be overridden by another pipeline.
• The workflow system does not allow conditional branches.
• Reader API has a rate limit of 20 request per minute (rpm) without API key. With an API key, the rate limit increases to 200 rpm.

Getting Started 🚀

We have a thorough guide on how to set up and get started with playscholar in our documentation.

Bugs or Requests 🐛

If you encounter any problems feel free to open an issue. If you feel the project is missing a feature, please raise a ticket on GitHub and I'll look into it. Pull requests are also welcome.

📫 How to reach me