Deploy PromptFlow to AWS

[abrichr]

Feature request

We wish to automate the deployment of https://microsoft.github.io/promptflow/how-to-guides/tune-prompts-with-variants.html to AWS.

e.g. under openadapt/server/promptflow.py (based on https://github.com/OpenAdaptAI/SoM/blob/main/deploy.py):

import os
import subprocess
import boto3
from loguru import logger
from pydantic_settings import BaseSettings
import fire

class Config(BaseSettings):
    AWS_ACCESS_KEY_ID: str
    AWS_SECRET_ACCESS_KEY: str
    AWS_REGION: str
    DOCKER_IMAGE_NAME: str = "web-classification-serve"  # Example Docker image name
    PROJECT_NAME: str
    OPENAI_API_KEY: str | None = None
    FLOW_SOURCE_DIR: str  # Path to your flow folder
    FLOW_OUTPUT_DIR: str = "./dist"  # Output directory for Docker format

    class Config:
        env_file = ".env"
        env_file_encoding = 'utf-8'

config = Config()

def build_flow_docker_format():
    """
    Builds the flow as a Docker format using the PromptFlow CLI.
    """
    try:
        cmd = f"pf flow build --source {config.FLOW_SOURCE_DIR} --output {config.FLOW_OUTPUT_DIR} --format docker"
        subprocess.run(cmd, check=True, shell=True)
        logger.info("Flow built as Docker format successfully.")
    except subprocess.CalledProcessError as e:
        logger.error(f"Failed to build flow as Docker format: {e}")

def build_docker_image():
    """
    Builds the Docker image from the Dockerfile generated by the flow build process.
    """
    try:
        cmd = f"docker build {config.FLOW_OUTPUT_DIR} -t {config.DOCKER_IMAGE_NAME}"
        subprocess.run(cmd, check=True, shell=True)
        logger.info("Docker image built successfully.")
    except subprocess.CalledProcessError as e:
        logger.error(f"Failed to build Docker image: {e}")

def deploy_docker_container():
    """
    Deploys the Docker container to an AWS EC2 instance.
    This method assumes an EC2 instance is already set up and configured to run Docker containers.
    """
    # This example skips EC2 setup details. Refer to the previous script for EC2 management logic.
    instance_ip = "YOUR_EC2_INSTANCE_IP"  # Placeholder for the instance IP
    docker_run_cmd = (
        f"docker run -d -p 80:8080 "
        f"-e OPEN_AI_CONNECTION_API_KEY='{config.OPENAI_API_KEY}' "
        f"{config.DOCKER_IMAGE_NAME}"
    )

    # SSH into the EC2 instance and run the docker container
    # Skipping SSH setup and command execution. Use Paramiko or similar library as in the previous script.
    logger.info("Docker container deployed successfully.")

class Deploy:

    @staticmethod
    def start() -> None:
        """
        Main method to orchestrate the build and deployment process.
        """
        build_flow_docker_format()
        build_docker_image()
        deploy_docker_container()
        logger.info("Flow deployment process completed.")

if __name__ == "__main__":
    fire.Fire(Deploy)

Motivation

To support fine tuning:

[abrichr]

Related, from https://repost.aws/questions/QUgLjWFHjxTIeHMFZMg-aLGg/managing-llm-prompt-flows:

I thought the combination of AWS StepFunctions and Amazon Bedrock might be similar to Promptflow. https://aws.amazon.com/jp/blogs/aws/build-generative-ai-apps-using-aws-step-functions-and-amazon-bedrock/ https://docs.aws.amazon.com/step-functions/latest/dg/sample-bedrock-prompt-chaining.html

[abrichr]

Related, from https://www.promptingguide.ai/techniques/ape:

• Prompt-OIRL: https://arxiv.org/abs/2309.06553
• OPRO: https://arxiv.org/abs/2309.03409 / https://github.com/google-deepmind/opro
• AutoPrompt: https://arxiv.org/abs/2010.15980
• Prefix Tuning: https://arxiv.org/abs/2101.00190
• Prompt Tuning: https://arxiv.org/abs/2104.08691

[abrichr]

Related, from https://www.promptingguide.ai/techniques/tot:

Tree of thought: https://github.com/princeton-nlp/tree-of-thought-llm / https://github.com/jieyilong/tree-of-thought-puzzle-solver

[abrichr]

From https://www.promptingguide.ai/techniques/activeprompt:

Active Prompt: https://arxiv.org/abs/2302.12246 / https://github.com/shizhediao/active-prompt