search

yohasebe / openai-chat-api-workflow

🎩 An Alfred 5 Workflow for using OpenAI Chat API to interact with GPT-4o 🤖💬 It also allows image generation 🖼️, image understanding 👀, speech-to-text conversion 🎤, and text-to-speech synthesis 🔈
https://github.com/yohasebe/openai-text-completion-workflow
MIT License
299 stars 8 forks source link
ai alfred chatbot dall-e gpt image-generation image-understanding openai speech-to-text text-to-speech whisper workflow

readme

OpenAI Chat API Workflow for Alfred

🎩 An Alfred 5 Workflow for using OpenAI Chat API to interact with GPT-4 🤖💬 It also allows image generation 🖼️, image understanding 👀, speech-to-text conversion 🎤, and text-to-speech synthesis 🔈

📦 Download OpenAI Chat API Workflow (version 3.2.0)

You can execute all the above features using:

The web UI is constructed by the workflow and runs locally on your Mac 💻 The API call is made directly between the workflow and OpenAI, ensuring your chat messages are not shared online with anyone other than OpenAI 🔒 Furthermore, OpenAI does not use the data from the API Platform for training 🚫

You can export the chat data to a simple JSON format external file 📄, and it is possible to continue the chat by importing it later 🔄

Installation

  1. Install Homebrew
  2. Run the following command in a terminal: brew install pandoc mpv sox jq duti
  3. Download and run OpenAI Chat API Workflow
  4. Set your OpenAI API key

Setup Hotkeys

You can set up hotkeys in the settings screen of the workflow. To set up hotkeys, double-click on the light purple workflow elements.

  1. Open Web UI (Recommended)
  2. Direct Query
  3. Send Selected Text
  4. Screen Capture for Image Understanding
  5. Speech to Text

Dependencies

To start using this workflow, you must set the environment variable apikey, which you can get by creating a new OpenAI account. See also the Configuration section below.

You will also need to install the pandoc and sox programs. Pandoc will allow this workflow to convert the Markdown response from OpenAI to HTML and display the result in your default web browser with syntax highlighting enabled (especially useful when using this workflow to generate program code). Sox will allow you to record voice audio to convert to text using Whisper speech-to-text API.

To set up dependencies (pandoc, mpv, sox, jq, and duti), first install homebrew. and run the following command.

  brew install pandoc mpv sox jq duti

Change Log

Recent Change Log

Complete Change Log

Methods of Execution

here are three methods to run the workflow: 1) Using commands within the Alfred UI, 2) Passing selected text to the workflow, 3) Utilizing the Web UI. Additionally, there’s a convenient method for making brief inquiries to GPT

Commands within the Alfred UI

You can enter a query text directly into Alfred textbox:

Passing Selected Text

You can select any text on your Mac and send it to the workflow:

Using Web Interface

You can open a web interface

Using the Default Browser

If your default browser is set to one of the following and the duti command is installed on your system, the web interface will automatically open in your chosen browser. If not, Safari will be used as the default.

Restart OpenAI Workflow server by executing openai-restart-server in case the web UI does not work as expected after changing the default browser.

Web UI Modes

Switch modes (light/dark/auto) with Web UI Mode selector in the settings.

Simple Direct Query/Chat

To quickly chat with GPT:

Basic Commands

With Direct Query, the input text is sent directly to the OpenAI Chat API as a prompt. You can also create a query by prepending or appending text to the input text.

Direct Query

The input text is directly sent as a prompt to the OpenAI Chat API.

Prepend Text + Query

After the initial text is entered, the user is prompted for additional text. The additional text is added before the initial text, and the resulting text is used as the query.

Append Text + Query

After the initial text is entered, the user is prompted for additional text. The additional text is added after the initial text and the resulting text is used as the query.

Generate Image

The DALL-E API (dall-e-3 or dall-e-2) is used to generate images according to the prompts entered. See Image Generation below.

Commands for Specific Purposes

Some of the examples shown on OpenAI's Examples page are incorporated into this Workflow as commands. Functions not prepared as commands can be realized by giving appropriate prompts to the above Basic Commands.

Write Program Code

GPT generates program code and example output according to the text entered. You can specify the purpose of the program, its function, the language and technology to be used, etc.

Example Input

Create a command line program that takes an English sentence and returns syntactically parsed output. Provide program code in Python and example usage.

Example Output

Ask in Your Language

You can ask questions in the language set to the variable first_language.

Note: If the value of first_language is not English (e.g. Japanese), the query may result in a more or less inaccurate response.

Translate L1 to L2

GPT translates text in the language specified in the variable first_language to the language specified in the second_language.

Translate L2 to L1

GPT translates text in the language specified in the variable second_language to the language specified in the variable first_language.

Grammar Correction

GPT corrects sentences that may contain grammatical errors. See OpenAI's description.

Brainstorm

GPT assists you in brainstorming innovative ideas based on any given text.

Create Study Notes

GPT provides study notes of a given topic. See OpenAI's description for this example.

Analogy Maker

GPT creates analogies. See OpenAI's description for this example.

Essay Outline

GPT generates an outline for a research topic. See OpenAI's description for this example.

TL;DR Summarization

GPT summarizes a given text. See OpenAI's description for this example.

Summarize for a 2nd Grader

GPT translates complex text into more straightforward concepts. See OpenAI's description for this example.

Keywords

GPT extracts keywords from a block of text. See OpenAI's description for this example.

Image Generation

The image generation can be executed through one of the above commands. It is also possible to use the web UI. By using the web UI, you can interactively change the prompt to get closer to the desired image.

When the image generation mode is set to dall-e-3, the user's prompt is automatically expanded to a more detailed and specific prompt. You can also edit the expanded prompt and regenerate the image.

Image Understanding

The image understanding can be executed through the openai-vision command. It starts a capture mode and lets you specify a part of the screen to be analyzed. Alternatively, you can specify an image file (jpg, jpeg, png, gif) using "OpenAI Vision" file action. This mode needs gpt-4o or gpt-4o-mini model to be set in the workflow settings.

Speech Synthesis and Speech Recognition

Most text-to-speech and speech-to-text features are available on the web UI. However, there are certain specific features that are provided as commands, such as audio file to text conversion and transcription with timestamps.

Text-to-Speech Synthesis

Text entered or response text from GPT can be read out in a natural voice using OpenAI's text-to-speech API.

Speech-to-Text Conversion

The Whisper API can convert speech into text in a variety of languages. Please refer to the Whisper API FAQ for available languages and other limitations.

Audio File to Text

You can select an audio file in mp3, mp4, flac, webm, wav, or m4a format (under 25MB) and send it to the workflow:

Record Voice Audio and Transcribe

You can record voice audio and send it to the Workflow for transcription using the Whisper API. The recording must be no longer than 30 minutes and will automatically stop after this time. Recording time is limited to 30 minutes and stops automatically after this limit.

transcript-srt

You can choose the format of the transribed text from text, srt or vtt in the workflow's settings. Below are examples in the text and srt formats:

transcript-srt

transcript-srt

Other Features

Import/Export

You can export your chat data to a straightforward JSON format file, and resume your conversation later by importing it back in.

To export data, simply click on Show Entire Chat in the chat window to navigate to the chat history page, then select Export Data. To import data, just hit Import Data on either the home page or the chat history page.

Monitor API Usage

To review your token usage for the current billing cycle on the OpenAI Usage Page, type the keyword openai-usage. For more details on billing, visit OpenAI's Billing Overview.

Configuration Parameters

You can set various parameters in the settings panel of this Workflow. Some of the parameters set here are used as default values but you can make temporary changes to the values on the web UI. You can also access the settings panel by clicking Open Config from the web UI.

Required Settings

Web UI Parameters

Chat Parameters

You may or may not use the following beta models. System prompt and parameter settings are not available for these models. Also, streaming is not supported for these model and the response time is longer than the other models.

Image Understading Parameters

Image Generation Parameters

Text-to-Speech Parameters

Speech-to-Text Parameters

Other Settings

Environment Variables

Environment variables can be accessed by clicking the [x] button located at the top right of the workflow settings screen. Normally, there is no need to change the values of the environment variables.

Author

Yoichiro Hasebe (yohasebe@gmail.com)

License

The MIT License

Disclaimer

The author assumes no responsibility for any potential damages arising from the use of this software.