gosaaan1 / hokulea-garage

AWS とTerraform を学ぶための箱庭
Apache License 2.0
1 stars 0 forks source link

Lambdaを活用したサーバレスアプリケーションの構築 #5

Open gosaaan1 opened 1 year ago

gosaaan1 commented 1 year ago

Goal

AWS Lambdaのメリット・デメリットを生かしたサーバレスアプリケーションについて考察してみる。

  1. どんな用途に向いている?
  2. 開発はしやすい?
  3. ビックデータの処理に使える?
  4. 運用・管理が面倒にならないか?
  5. コスパはEC2より有利?
  6. モノリスなアプリとの違いは?

AI


gosaaan1 commented 1 year ago

サンプルアプリづくり。せっかくならデータ収集&分析を行うようなものが作れるといいな。

元ネタ

gosaaan1 commented 1 year ago

AWS SAM のインストール

https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/install-sam-cli.html

hokulea-garage/aws_sam_project on  main [#!?] 
⬢ [Docker] ❯ brew install aws/tap/aws-sam-cli

hokulea-garage/aws_sam_project on  main [#!?] took 27s 
⬢ [Docker] ❯ sam --version
SAM CLI, version 1.80.0

チュートリアル:Hello World アプリケーションのデプロイ

https://docs.aws.amazon.com/ja_jp/serverless-application-model/latest/developerguide/serverless-getting-started-hello-world.html

サンプルアプリケーションのダウンロード(途中)

hokulea-garage/aws_sam_project on  main [#!?] 
⬢ [Docker] ❯ sam init

        SAM CLI now collects telemetry to better understand customer needs.

        You can OPT OUT and disable telemetry collection by setting the
        environment variable SAM_CLI_TELEMETRY=0 in your shell.
        Thanks for your help!

        Learn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html

You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Multi-step workflow
        3 - Serverless API
        4 - Scheduled task
        5 - Standalone function
        6 - Data processing
        7 - Hello World Example With Powertools
        8 - Infrastructure event management
        9 - Serverless Connector Hello World Example
        10 - Multi-step workflow with Connectors
        11 - Lambda Response Streaming
        12 - Lambda EFS example
        13 - DynamoDB Example
        14 - Machine Learning
Template: 1

Use the most popular runtime and package type? (Python and zip) [y/N]: y

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: y
X-Ray will incur an additional cost. View https://aws.amazon.com/xray/pricing/ for more details

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: y
AppInsights monitoring may incur additional cost. View https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/appinsights-what-is.html#appinsights-pricing for more details

Project name [sam-app]: 

Cloning from https://github.com/aws/aws-sam-cli-app-templates (process may take a moment)

    -----------------------
    Generating application:
    -----------------------
    Name: sam-app
    Runtime: python3.9
    Architectures: x86_64
    Dependency Manager: pip
    Application Template: hello-world
    Output Directory: .
    Configuration file: sam-app/samconfig.toml

    Next steps can be found in the README file at sam-app/README.md

Commands you can use next
=========================
[*] Create pipeline: cd sam-app && sam pipeline init --bootstrap
[*] Validate SAM template: cd sam-app && sam validate
[*] Test Function in the Cloud: cd sam-app && sam sync --stack-name {stack-name} --watch

こんな感じでファイルが作られます… image

RuntimeでPython3.9を選んだので、Python3.9を入れようとするのだけど失敗。

hokulea-garage/aws_sam_project/sam-app on  main [#!?] via 🐍 v3.10.6 took 7s 
⬢ [Docker] ❯ pyenv install 3.9
Downloading Python-3.9.16.tar.xz...
-> https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tar.xz
Installing Python-3.9.16...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/root/.pyenv/versions/3.9.16/lib/python3.9/ssl.py", line 99, in <module>
    import _ssl             # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems

BUILD FAILED (Ubuntu 22.04 using python-build 20180424)

Inspect or clean up the working tree at /tmp/python-build.20230418224911.14481
Results logged to /tmp/python-build.20230418224911.14481.log

Last 10 log lines:
                $ensurepip --root=/ ; \
fi
Looking in links: /tmp/tmpglsr6aoa
Processing /tmp/tmpglsr6aoa/setuptools-58.1.0-py3-none-any.whl
Processing /tmp/tmpglsr6aoa/pip-22.0.4-py3-none-any.whl
Installing collected packages: setuptools, pip
  WARNING: The scripts pip3 and pip3.9 are installed in '/root/.pyenv/versions/3.9.16/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.0.4 setuptools-58.1.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

brewをインストールしているときはこうしないといけないっぽい…

CFLAGS="-I$(brew --prefix openssl)/include" LDFLAGS="-L$(brew --prefix openssl)/lib" pyenv install -v 3.9

hokulea-garage on  main [#] 
⬢ [Docker] ❯ pyenv versions
* system (set by /root/.pyenv/version)
  3.9.16

AWS SAMのプロジェクトに Python 3.9.16 を設定。

hokulea-garage on  main [#!?] 
⬢ [Docker] ❯ cd aws_sam_project/

hokulea-garage/aws_sam_project on  main [#!?] 
⬢ [Docker] ❯ pyenv local 3.9.16

build を実行。

hokulea-garage/aws_sam_project/sam-app on  main [#!?] via 🐍 v3.9.16 
⬢ [Docker] ❯ sam build
Starting Build use cache
Manifest file is changed (new hash: 3298f13049d19cffaa37ca931dd4d421) or dependency folder (.aws-sam/deps/402b00c8-863f-4f99-b259-0e94de672cfa) is missing for (HelloWorldFunction), downloading dependencies and copying/building source
Building codeuri: /workspaces/hokulea-garage/aws_sam_project/sam-app/hello_world runtime: python3.9 metadata: {} architecture: x86_64 functions: HelloWorldFunction
Running PythonPipBuilder:CleanUp
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Running PythonPipBuilder:CopySource

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided

アプリケーションをAWSにデプロイする

★デプロイする前に確認しておきたいこと。

gosaaan1 commented 1 year ago

アプリケーションをローカルでテストする

APIをローカルでホストするにはDockerが必要。

hokulea-garage/aws_sam_project/sam-app on  main [#!?] via 🐍 v3.9.16 
⬢ [Docker] ❯ sam local start-api
Error: Running AWS SAM projects locally requires Docker. Have you got it installed and running?

TerraformのコンテナではDockerは動かせないので、WSLに切り替える。 brewが入っていたので、samをWSLに入れて実行。

$ sam local start-api

curlの実行結果。

$ curl http://127.0.0.1:3000/hello
{"message": "hello world"}
gosaaan1 commented 1 year ago

アプリケーションを AWS クラウド にデプロイ/クリーンアップする

チュートリアルのとおりに進めればよいのだけど、わかりづらいところだけ補足。

実行するコマンド:aws-vaultを使っているときはこう。

aws-vault exec hokulea -- sam deploy --guided

画面に表示されるプロンプトに従ってください。インタラクティブな形式で提供されるデフォルトオプションを受け入れるには、Enter を押して応答します。

以下だけyで応答する必要あり。

HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y

デプロイされたAWS CloudFormationスタックは管理コンソールで確認することができる。 スクリーンショット 2023-05-04 160611

gosaaan1 commented 1 year ago

コンテナイメージを使ったLambda関数を使う場合

hokulea-garage/aws_sam_project on  main [#!?] via 🐍 v3.9.16 
⬢ [Docker] ❯ sam init

You can preselect a particular runtime or package type when using the `sam init` experience.
Call `sam init --help` to learn more.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
        1 - Hello World Example
        2 - Multi-step workflow
        3 - Serverless API
        4 - Scheduled task
        5 - Standalone function
        6 - Data processing
        7 - Hello World Example With Powertools
        8 - Infrastructure event management
        9 - Serverless Connector Hello World Example
        10 - Multi-step workflow with Connectors
        11 - Lambda Response Streaming
        12 - Lambda EFS example
        13 - DynamoDB Example
        14 - Machine Learning
Template: 1

Use the most popular runtime and package type? (Python and zip) [y/N]: N

Which runtime would you like to use?
        1 - aot.dotnet7 (provided.al2)
        2 - dotnet6
        3 - dotnet5.0
        4 - dotnetcore3.1
        5 - go1.x
        6 - go (provided.al2)
        7 - graalvm.java11 (provided.al2)
        8 - graalvm.java17 (provided.al2)
        9 - java11
        10 - java8.al2
        11 - java8
        12 - nodejs18.x
        13 - nodejs16.x
        14 - nodejs14.x
        15 - nodejs12.x
        16 - python3.9
        17 - python3.8
        18 - python3.7
        19 - python3.10
        20 - ruby2.7
        21 - rust (provided.al2)
Runtime: 16

What package type would you like to use?
        1 - Zip
        2 - Image
Package type: 2

Based on your selections, the only dependency manager available is pip.
We will proceed copying the template using pip.

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: 

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: 

Project name [sam-app]: simple-flask-app

    -----------------------
    Generating application:
    -----------------------
    Name: simple-flask-app
    Base Image: amazon/python3.9-base
    Architectures: x86_64
    Dependency Manager: pip
    Output Directory: .
    Configuration file: simple-flask-app/samconfig.toml

    Next steps can be found in the README file at simple-flask-app/README.md

Commands you can use next
=========================
[*] Create pipeline: cd simple-flask-app && sam pipeline init --bootstrap
[*] Validate SAM template: cd simple-flask-app && sam validate
[*] Test Function in the Cloud: cd simple-flask-app && sam sync --stack-name {stack-name} --watch
gosaaan1 commented 1 year ago

Pythonの場合、AWSGIを使用すると、Flask や Django などの WSGI 互換のミドルウェアとフレームワークをAWS API Gateway/Lambda プロキシ統合で使用できます。

※以下はFlaskRESTfulもあわせて導入した例。

ソース

requirements.txt

requests
Flask
aws-wsgi
flask-restful

app.py

import awsgi
from flask import Flask, request
from flask_restful import Resource, Api

app = Flask(__name__)
api = Api(app)

class HelloWorld(Resource):
    def get(self):
        return {'msg': 'get method'}

    def post(self):
        return {'msg': 'post method'}

api.add_resource(HelloWorld, '/hello')

def lambda_handler(event, context):
    return awsgi.response(app, event, context)

template.yaml

      Events:
        ApiRoot:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: '/'
            Method: ANY
        ApiProxy:
          Type: Api
          Properties:
            Path: '/{proxy+}'
            Method: ANY

実行結果

hokulea-garage/aws_sam_project/simple-flask-app on  main [$!?] via 🐍 v3.9.16 took 44m23s 
❯ sam build
Building codeuri: /home/hokulea/hokulea-garage/aws_sam_project/simple-flask-app runtime: None metadata: {'Dockerfile': 'Dockerfile', 'DockerContext': '/home/hokulea/hokulea-garage/aws_sam_project/simple-flask-app/flask_app', 'DockerTag': '1.0.0'} architecture: x86_64 functions: FlaskAppFunction
Building image for FlaskAppFunction function
Setting DockerBuildArgs: {} for FlaskAppFunction function
Step 1/5 : FROM public.ecr.aws/lambda/python:3.9
 ---> 792fdc65bb1f
Step 2/5 : COPY requirements.txt ./
 ---> Using cache
 ---> 4a6b401946db
Step 3/5 : RUN python3.9 -m pip install -r requirements.txt -t .
 ---> Using cache
 ---> 3e9326821d5a
Step 4/5 : COPY app.py ./
 ---> 145937adb3b9
Step 5/5 : CMD ["app.lambda_handler"]
 ---> Running in f6489d47da50
Removing intermediate container f6489d47da50
 ---> 140248f2f3ce
Successfully built 140248f2f3ce
Successfully tagged flaskappfunction:1.0.0

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided

hokulea-garage/aws_sam_project/simple-flask-app on  main [$!?] via 🐍 v3.9.16 
❯ sam local start-api
Initializing the lambda functions containers.
Building image.................
Using local image: flaskappfunction:rapid-x86_64.

Containers Initialization is done.
Mounting FlaskAppFunction at http://127.0.0.1:3000/{proxy+} [DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT]
Mounting FlaskAppFunction at http://127.0.0.1:3000/ [DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. If you used sam build before running local commands, you will need to re-run sam build for the changes to be picked up. You only need to restart SAM CLI if you update your AWS SAM template
2023-05-06 09:40:40 WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:3000
2023-05-06 09:40:40 Press CTRL+C to quit
Invoking Container created from flaskappfunction:1.0.0
Reuse the created warm container for Lambda function 'FlaskAppFunction'
Lambda function 'FlaskAppFunction' is already running
START RequestId: 4aed1aaa-1c0b-492f-88ba-0a8be76ec805 Version: $LATEST
2023-05-06 09:41:07 127.0.0.1 - - [06/May/2023 09:41:07] "GET /hello HTTP/1.1" 200 -
END RequestId: 4aed1aaa-1c0b-492f-88ba-0a8be76ec805
REPORT RequestId: 4aed1aaa-1c0b-492f-88ba-0a8be76ec805   Init Duration: 0.22 ms  Duration: 569.23 ms      Billed Duration: 570 ms Memory Size: 128 MB      Max Memory Used: 128 MB
Invoking Container created from flaskappfunction:1.0.0
Reuse the created warm container for Lambda function 'FlaskAppFunction'
Lambda function 'FlaskAppFunction' is already running
START RequestId: 71e15023-02b7-4b40-b7f2-4d265168000c Version: $LATEST
2023-05-06 09:41:21 127.0.0.1 - - [06/May/2023 09:41:21] "POST /hello HTTP/1.1" 200 -
END RequestId: 71e15023-02b7-4b40-b7f2-4d265168000c
REPORT RequestId: 71e15023-02b7-4b40-b7f2-4d265168000c  Duration: 1.61 ms     Billed Duration: 2 ms   Memory Size: 128 MB     Max Memory Used: 128 MB
hokulea-garage/aws_sam_project/simple-flask-app on  main [$!?] via 🐍 v3.9.16 
❯ curl http://127.0.0.1:3000/hello
{"msg": "get method"}

hokulea-garage/aws_sam_project/simple-flask-app on  main [$!?] via 🐍 v3.9.16 took 2s 
❯ curl -X POST http://127.0.0.1:3000/hello 
{"msg": "post method"}
gosaaan1 commented 1 year ago

【新機能】Amazon API Gatewayの設定方法にcatch-allパス変数、ANYメソッド、Lambdaとの新しいプロキシ連携の3機能が追加。

gosaaan1 commented 1 year ago
gosaaan1 commented 1 year ago

クリーンアップするときのコマンド: 何らかの要因でデプロイが失敗してしまった時も、このコマンドで一旦AWS CloudFormationスタックを削除する必要がある。

aws-vault exec hokulea -- aws cloudformation delete-stack --stack-name sam-app --region ap-northeast-1

これ、やっちゃうと次回のsam deployが失敗します。 正しくは

aws-vault exec hokulea -- sam delete --stack-name sam-app
gosaaan1 commented 1 year ago
hokulea-garage/aws_sam_project/simple-flask-app on  main [$!?] via 🐍 v3.9.16 
❯ curl http://127.0.0.1:3000/hello
{"msg": "get method"}

hokulea-garage/aws_sam_project/simple-flask-app on  main [$!?] via 🐍 v3.9.16 took 2s 
❯ curl -X POST http://127.0.0.1:3000/hello 
{"msg": "post method"}

デプロイすると動かなくなる。(以下はLambda関数のログ) Lambdaは/tmpのみ書き込みを許可している。

[ERROR] OSError: [Errno 30] Read-only file system: '/var/task/instance'
Traceback (most recent call last):
File "/var/lang/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/var/task/app.py", line 10, in <module>
db.init_app(app)
File "/var/task/flask_sqlalchemy/extension.py", line 325, in init_app
self._apply_driver_defaults(options, app)
File "/var/task/flask_sqlalchemy/extension.py", line 576, in _apply_driver_defaults
os.makedirs(app.instance_path, exist_ok=True)
File "/var/lang/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
gosaaan1 commented 1 year ago

デプロイすると動かなくなる。(以下はLambda関数のログ)

RDSのクライアントをきちんと設定すれば、エラーは出なくなりましたとさ。 ただ、"ローカルとLambdaとのパーミッションの違いで動かなくなる可能性"があることだけ覚えておこう…

修正

aws_sam_project/simple-flask-app/simple_flask/requirements.txt

pymysql

aws_sam_project/simple-flask-app/simple_flask/app.py

app.config["SQLALCHEMY_DATABASE_URI"] = "mysql+pymysql://hoge:hoge@hogehoge.hogehoge.ap-northeast-1.rds.amazonaws.com/hogedb?charset=utf8mb4"

結果

https://hogehoge.execute-api.ap-northeast-1.amazonaws.com/Prod/hello [GET]

{"msg": "get method"}
gosaaan1 commented 1 year ago

デプロイするごとにコンテナイメージが増えていくのは、ちょっと気に留めておかないといけない。 (じわりとコストがかさんでいくところ) image

gosaaan1 commented 1 year ago

API Gateway API へのアクセスの制御

gosaaan1 commented 1 year ago

AWS LambdaでAPI開発するときのパターン集

gosaaan1 commented 1 year ago

Bingのチャットを使ってテンプレートを作ってみた。使えるのかな…

Cognito を使った AWS SAM テンプレートの例についてお探しですね。Web 検索の結果に基づいて、以下のような回答ができます。

AWS SAM テンプレートでは、ApiAuth データ型を使って Amazon Cognito ユーザープールを定義することで、API へのアクセスを制御できます12。 例えば、次のようなテンプレートセクションがあります12

Resources:
  MyApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      Cors: "'*'"
      Auth:
        DefaultAuthorizer: MyCognitoAuthorizer
        Authorizers:
          MyCognitoAuthorizer:
            UserPoolArn: !GetAtt MyCognitoUserPool.Arn
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./src
      Handler: lambda.handler
      Runtime: nodejs12.x
      Events:
        Root:
          Type: Api
          Properties:
            RestApiId: !Ref MyApi
            Path: /
            Method: GET
  MyCognitoUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: !Ref CognitoUserPoolName
      Policies:
        PasswordPolicy:
          MinimumLength: 8
      UsernameAttributes:
        - email
      Schema:
        - AttributeDataType: String
          Name: email
          Required: false
  MyCognitoUserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    Properties:
      UserPoolId: !Ref MyCognitoUserPool
      ClientName: !Ref CognitoUserPoolClientName
      GenerateSecret: false

このテンプレートでは、MyApi という名前の API が作成され、MyCognitoAuthorizer という名前の Cognito ユーザープールをオーソライザーとして使用します。MyFunction という名前の Lambda 関数は、GET メソッドで / パスにアクセスしたときに呼び出されます。MyCognitoUserPool と MyCognitoUserPoolClient は、ユーザープールとそのクライアントを作成するためのリソースです。 Amazon Cognito ユーザープールの詳細については、API Gateway デベロッパーガイド の「 Amazon Cognito ユーザープールをオーソライザーとして使用して REST API へのアクセスを制御する 」を参照してください12。 これでお役に立てましたでしょうか?😊

gosaaan1 commented 1 year ago
gosaaan1 commented 1 year ago

RDS/DynamoDB/S3などストレージサービスをsam local start-apiで使う場合 (DRAFT)

gosaaan1 commented 1 year ago

少し間が空いてしまったので進め方を整理する。

gosaaan1 commented 1 year ago

SAM パイプラインの作成

コマンドのヘルプ

❯ aws-vault exec hokulea -- sam pipeline init --help
Usage: sam pipeline init [OPTIONS]

  This command generates a pipeline configuration file that your CI/CD system
  can use to deploy serverless applications using AWS SAM.

  Before using sam pipeline init, you must bootstrap the necessary resources
  for each stage in your pipeline. You can do this by running sam pipeline
  init --bootstrap to be guided through the setup and configuration file
  generation process, or refer to resources you have previously created with
  the sam pipeline bootstrap command.

Options:
  --config-env TEXT   Environment name specifying default parameter values in
                      the configuration file.  [default: default]
  --config-file TEXT  Configuration file containing default parameter values.
                      [default: samconfig.toml]
  --bootstrap         Enable interactive mode that walks the user through
                      creating necessary AWS infrastructure resources.
  --debug             Turn on debug logging to print debug message generated
                      by AWS SAM CLI and display timestamps.
  -h, --help          Show this message and exit.

ワークショップとの相違点

aws-vaultを使っているときは認証情報は変数で渡されるので1

[2] Account details
The following AWS credential sources are available to use.
To know more about configuration AWS credentials, visit the link below:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html                
        1 - Environment variables
        2 - default (named profile)
        3 - hokulea (named profile)
        q - Quit and configure AWS credentials
Select a credential source to associate with this stage: 1

creationのところでエラーが出て止まってしまう。

This will create the following required resources for the 'dev' configuration: 
        - Pipeline IAM user
        - Pipeline execution role
        - CloudFormation execution role
        - Artifact bucket
        - ECR image repository
Should we proceed with the creation? [y/N]: y
        Creating the required resources...
Error: Failed to create managed resources: Waiter StackCreateComplete failed: Waiter encountered a terminal failure state: For expression "Stacks[].StackStatus" we matched expected path: "ROLLBACK_COMPLETE" at least once
gosaaan1 commented 1 year ago

--debugオプションをつけたときの実行結果。

This will create the following required resources for the 'dev' configuration: 
        - Pipeline IAM user
        - Pipeline execution role
        - CloudFormation execution role
        - Artifact bucket
        - ECR image repository
Should we proceed with the creation? [y/N]: y
        Updating the required resources...
2023-06-14 19:08:11,412 | Managed S3 stack [aws-sam-cli-managed-dev-pipeline-resources] not found. Creating a new one.
        Creating the required resources...
2023-06-14 19:08:11,495 | Failed to create managed resources
Traceback (most recent call last):
  File "samcli/lib/utils/managed_cloudformation_stack.py", line 194, in _create_or_update_stack
  File "samcli/lib/utils/managed_cloudformation_stack.py", line 257, in _create_stack
  File "botocore/client.py", line 530, in _api_call
  File "botocore/client.py", line 960, in _make_api_call
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the CreateChangeSet operation: Stack [aws-sam-cli-managed-dev-pipeline-resources] already exists and cannot be created again with the changeSet [InitialCreation].
2023-06-14 19:08:11,495 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2023-06-14 19:08:11,506 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': 'b1eb7dd8-ef13-4297-b700-d989d7131404', 'installationId': 'f5602809-baf9-4f02-9221-ed6d51a43187', 'sessionId': '0c9f1639-045f-4f35-8574-4b741f98dc8e', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.7.10', 'samcliVersion': '1.82.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam pipeline init', 'metricSpecificAttributes': {'projectType': 'CFN', 'gitOrigin': None, 'projectName': '64a6fdd92bdee2bf3b2c3f45fecc8da2f23f2caa75de25b676a91a9125efc53d', 'initialCommit': None}, 'duration': 51989, 'exitReason': 'ManagedStackError', 'exitCode': 1}}]}
2023-06-14 19:08:12,075 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: Failed to create managed resources: An error occurred (ValidationError) when calling the CreateChangeSet operation: Stack [aws-sam-cli-managed-dev-pipeline-resources] already exists and cannot be created again with the changeSet [InitialCreation].
gosaaan1 commented 1 year ago

CloudFormation側 image

gosaaan1 commented 1 year ago

https://qiita.com/rm0063vpedc15/items/9971218e79ba6616c8b5#%E3%82%A8%E3%83%A9%E3%83%BC%E5%86%85%E5%AE%B9

2023/4にあったS3のデフォルト設定の変更が関係しているものと思っています。

gosaaan1 commented 1 year ago

以下バージョンで解決。

❯ sam --version
SAM CLI, version 1.86.1
gosaaan1 commented 1 year ago

AWS SAM パイプラインの導入: サーバーレスアプリケーションのデプロイパイプラインを自動的に生成する

gosaaan1 commented 1 year ago

GitHub Actionを用いたCI/CDについてメモ書き程度でまとめた。 https://github.com/gosaaan1/hokulea-garage/wiki/CI-CD-with-GitHub

gosaaan1 commented 1 year ago

GitHub Action か CodePipelineか

参考:Github Actions 2年使ってみてわかったことまとめ

gosaaan1 commented 1 year ago

AWS ToolKit for VSCodeを入れてみる

gosaaan1 commented 1 year ago

IAM Identity Centerのあたりの手順を端折ってしまったけど、(プログラムによるアクセス認証あたりが参考になる)AWS ToolKit for VSCodeは使えるようになった。 手順の数が多いのでIaCにできればよいのだけど、ルートアカウントを使う操作なので無理そう… image

gosaaan1 commented 1 year ago

サーバーレスアプリケーションの操作

gosaaan1 commented 1 year ago

sam local start-apiで他のコンテナ(例えばMySQL)に接続する場合は以下のようにすればよい。

※ヒントになった記事:Container cannot be connected to host network

gosaaan1 commented 1 year ago

テーブルのマイグレーション(db.create_all())が動かない等はあるが、FlaskのTodoアプリが普通に動いている。

image