This is a WebUI extension to help users migrate existing workload (inference, train, ckpt merge, etc.) from local server or standalone server to AWS Cloud.
If you have not installed the Stable Diffusion WebUI, or the proper version of the extensions this project supports, please refer to this step for setup.
Bucket
is the bucket to store all your solution assets, Email
is the mail address you register to receive notification for events like model training complete, the SdExtensionApiKey
is the basic authentication for your api url connection:For users who need explicit IAM permissions for strict account control, we provide CloudFormation template to help system administrators create IAM roles with minimal permissions, user can create & delete middleware CloudFormation templates through newly such created roles.
Create new minimum role first, e.g. sd-min-role.
Specify such role in Cloudformation creation.
Notice : We prefer to deploy our solution in us-east-1 region, the reason is that in another region there is an existing S3 CORS issue which will block user to upload inference config for around 2 hours. That mean user need to wait around 2 hours after deploy the middleware to do the inference job. We will keep monitoring the progress of this issue.
Prerequisites To set up the development environment, you will need to have AWS account and tools with a preferred version below to install from source code:
Deploy the project to your AWS account (make sure your current profile has Administrator access, with AWS CDK, Docker installed):
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
cd infrastructure/
npm install
npx cdk bootstrap
npx cdk deploy
You can specify the following three parameters:
npx cdk deploy --parameters api-token=<XXXXXXX>\
--parameters sub-email-address=<YOUREMAIL@XXX.COM>\
--parameters util-instance-type=<ml.r5.large | ml.r5.xlarge| ml.c6i.2xlarge | ml.c6i.4xlarge | ml.r5.large>
The project build and deployment will take about 25 minutes, and the first time can be longer due to container image packaging. Once the project is deployed successfully to your AWS account, you will see output similar below:
Stable-diffusion-aws-extension-middleware-stack: creating CloudFormation changeset...
✅ Stable-diffusion-aws-extension-middleware-stack
✨ Deployment time: 218.39s
Outputs:
Stable-diffusion-aws-extension-middleware-stack.ApiGatewayUrl = {API_GATEWAY_URL} // this url will needed later
Stable-diffusion-aws-extension-middleware-stack.S3BucketName = {YOUR_S3_BUCKET_LOCATION}
Stable-diffusion-aws-extension-middleware-stack.SNSTopicName = {SNS_TOPIC}
Stack ARN:
arn:aws:cloudformation:us-east-1:{ACCOUNT_NUMBER}:stack/Stable-diffusion-aws-extension-middleware-stack/{ARN}
Go back the WebUI console, and choose "AWS SageMaker" tab, enter the API Gateway URL and API token you just got from CloudFormation/CDK output, click "Update Setting" and "Test Connection" button to make sure the connection is successful. Or you can create new file called sagemaker_ui.json
for the same purpose, the file should be placed under stable diffusion root folder and the content should be like below:
{
"api_gateway_url": "{API_GATEWAY_URL}",
"api_token": "{API_GATEWAY_KEY}"
}
Please refer to user guide for following detailed operations.
Stable Diffusion WebUI is a popular open-source GitHub project that provides an intuitive and user-friendly interface for data scientists and developers to interact with pre-trained txt2img/img2img model. The project has gained traction in the community (forks/stars/prs) for its ability to streamline the process of training, evaluating, and deploying models. As the demand for scalable and efficient machine learning solutions continues to rise, the Stable Diffusion WebUI project has emerged as a go-to tool for many user. Some user existing workflow is shown below that the data scientists had to jump from customized WebUI, EC2 and manual scripts to accomplished a single model finetune process, which are:
Thus we plan to contribute a solution aiming to mitigate the above issue and provide a lite, decouple and user-friendly AWS SageMaker based extension, in response to this growing demand and specific user requirement. We integrate WebUI with AWS SageMaker, a fully managed service that allows users to build, train, and deploy machine learning models in the cloud. This extension will enable users to leverage the power of AWS SageMaker as the backend for model training and inference. With the AWS customized extension in place, Stable Diffusion WebUI will offer its users a more streamlined and cost-effective solution to optimize their existing workflows, including model training/finetune, inference and iteration with fast community pace.
We have provided a Stable Diffusion WebUI extension and AWS middleware for user, and user will install such an extension by importing provided GitHub URL and AWS middleware by launch offered CloudFormation template in AWS console. Brief user tutorial will be: User will first install the extension, extra tab will be installed for user to manager AWS credential, building AWS native SD model etc. then user will navigate to original txt2img tab, configure setting like CFG scale, batch count/size etc., then click 'Generate on Cloud' button to get the generated image. Thus, providing user another alternative to utilize cloud resource without break existing user experience. Please refer to user guide for more details.
Diagram below is the brief view of internal workflow between our extension and middleware, user will keep launching community WebUI onto standalone EC2/local server with our extension installed, while the training and inference part will be passed onto AWS cloud (SageMaker, S3 etc.) through the RESTful API provided by middleware installed on user’s AWS account. Note the middleware is per AWS account, means it could be installed separately as work node to communicate with WebUI as control node, user only need to input endpoint URL and API key per account to decide which specific AWS account will be used for successive jobs.
The major function of middleware is to provide RESTful API for WebUI extension to interact with AWS cloud resource (SageMaker, S3 etc.) with OpenAPI conformant schema, it will handling the request authentication, request dispatch to specific SageMaker/S3 API (SageMaker.jumpstart/model/predictor/estimator/tuner/utils etc.) and model lifecycle. Diagram below is the overall architecture of middleware, including API Gateway and Lambda to fulfill the RESTful API function and Step Function to orchestrate the model lifecycle.
.
├── CHANGELOG.md
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── NOTICE
├── README.md
├── THIRD-PARTY-LICENSES.txt
├── build_scripts -- scripts to build the docker images, we use these scripts to build docker images on cloud
├── buildspec.yml -- buildspec file for CodeBuild, we have code pipeline to use this buildspec to transfer the CDK assets to Cloudformation templates
├── deployment -- scripts to deploy the CloudFormation template
├── docs
├── infrastructure -- CDK project to deploy the middleware, all the middle ware infrastructure code is in this directory
├── install.py -- install dependencies for the extension
├── install.sh -- script to set the webui and extension to specific version
├── javascript -- javascript code for the extension
├── middleware_api -- middleware api denifition and lambda code
├── sagemaker_entrypoint_json.py -- wrapper function for SageMaker
├── scripts -- extension related code for WebUI
└── utils.py -- wrapper function for configure options
Check our wiki for the latest & historical version