Summarize text using ChatGPT or a local LLM, with support for multiple large text files, PDF files and translation.
| Area | Feature |
|---|---|
| File types | - Summarize text, markdown, HTML, PDF files |
| Summarization levels | - Summarize at different lavels: short, long, and per-paragraph |
| Translation | - Translate to a target language |
| Data sources | - Batch summarize whole directories of files - Download a file via URL and summarize it |
| Private LLM | - Optionally use a locally hosted LLM, for maximum privacy and prevent any loss of IP (Intellectual Property) |
| Cost savings | - Avoid re-summarizing a previously processed file - Calculate cost estimates (when using Open AI) |
| Output files | - Output files in YAML format (as opposed to JSON): cheaper for LLM to generate, easy for humans to read - Output files with a “.yaml.txt” file extension, for easy previewing and search in storage tools like Dropbox or SharePoint or Google Drive |
If running a local LLM:
If using Open AI Chat GPT:
To see the available options:
./go.shor
python3 main_cli.pyOutput:
Usage: main_cli.py <path to input file or input directory or URL> [options]
The options are:
[-l --language - The target output language. The default is set in config.py]
[-o --output - The output directory. By default is None, so output is to stdout (no files are output).]
[-h --help]data/input.txt.Tip: unless using a local LLM, make sure the text does not contain commercially or personally sensitive information!
go.sh script:./go.sh data/input.txt [options]
To summarize different file(s):
python3 main_cli.py <path to input text file or directory> [options]
gpt-summarizer can also summarize PDF files:
python3 main_cli.py <path to PDF file or directory> [options]
The output is printed to STDOUT (terminal output):
=== === === [1] Summarizing './data/input.txt' === === ===
Summarizing file at './data/input.txt' into English...
=== === === [2] Short Summary = Chunk 1 of 1 === === ===
The study examines how language models perform with long contexts, finding that they struggle when relevant information is in the middle of the input. Performance decreases as context length increases, even for models designed for long contexts, offering insights for future model evaluation.
=== === === [3] FULL Short Summary === === ===
The study examines how language models perform with long contexts, finding that they struggle when relevant information is in the middle of the input. Performance decreases as context length increases, even for models designed for long contexts, offering insights for future model evaluation.
=== === === [4] FULL Long Summary === === ===
The research delves into the performance of language models when processing long contexts, revealing that models face challenges when relevant information is located in the middle of the input. As the context length grows, performance diminishes, impacting tasks like multi-document question answering and key-value retrieval. This study sheds light on how language models utilize input contexts and proposes new evaluation methods for forthcoming long-context models.
=== === === [5] FULL paragraphs Summary === === ===
Recent language models can handle long contexts but struggle with utilizing longer context effectively.
Performance is highest when relevant information is at the beginning or end of the input context.
Models face significant degradation when required to access relevant information in the middle of long contexts.
Performance decreases as the input context length increases, even for models explicitly designed for long contexts.
The study offers insights into how language models utilize input context and suggests new evaluation protocols for future long-context models.
-- THIS FILE time: 0:00:05s
-- THIS FILE estimated cost: $0.0006715
=== === === [6] Completed === === ===
1 files processed in 0:00:05s
-- Total estimated cost: $0.0006715See also an example of summarizing this README.
Large files are broken into chunks for processing, with a single concatenated final output.
Costs are estimated using the figures in config.py.
If an output directory is specified as an option, then each input file has an equivalent output file, in YAML format.
gpt-summary can be used in 2 ways:
1 - via remote LLM on Open-AI (Chat GPT) 2 - OR via local LLM (see the model types supported by ctransformers).
First, edit config.py according to whether you can use GPU acceleration:
pip install cornsnake==0.0.60 html2text==2024.2.26 json5==0.9.25 ollama==0.2.0 openai==1.23.6 PyMuPDF==1.24.1 pyyaml==6.0.1 ruff==0.3.5Get an Open AI key
Set environment variable with your Open AI key:
export OPENAI_API_KEY="xxx"Add that to your shell initializing script (~/.zprofile or similar)
Load in current terminal:
source ~/.zprofileSet the value of LOCAL_CTRANSFORMERS_MODEL_FILE_PATH to be "".
Set the value of OLLAMA_MODEL_NAME to be "".
ollama pull llama3ollama serveEdit config.py - set OLLAMA_MODEL_NAME to the name of the model from step 2
Set the value of LOCAL_CTRANSFORMERS_MODEL_FILE_PATH to be "".
YAML is generally cheaper and faster, but some LLMs may be more reliable if asked to output JSON.
For a local LLM, you can decide which format to use via config.py:
is_local__json_not_yamlpip3 install --upgrade ctransformers pymupdfQuality models are available at hugging face - see TheBloke.
Example: https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/llama-2-13b-chat.ggmlv3.q4_0.bin
OR via bash:
./download-llama-2-13B-model.shSet LOCAL_CTRANSFORMERS_MODEL_FILE_PATH to the path to the model file.
Set OLLAMA_MODEL_NAME to be "".
mitigations:
Inspired by an excellent DeepLearning.ai course: Prompt Engineering for Developers