Welcome to IPFS Cluster on ECS Fargate!

A CloudFormation template (ipfscluster-cf-final.template) and a python CDK script to deploy IPFS and IPFS Cluster on ECS Fargate.

Pleare refer to the AWS Blog for more details.

IMPORTANT

The default CDK stack will deploy and run the default ipfs/kubo:latest and ipfs/ipfs-cluster:latest Docker images.

If killed, the IPFS daemon leaves a repo.lock file that will prevent the restart of the Fargate task.

Running your own custom image FROM ipfs/kubo is required to fix this issue.

We've created a Dockerfile_efs just for that. Use it to build your own Docker image and reference it in your CloudFormation or CDK stack.

Note: Dockerfile_s3 is also available to build a Docker image that sets up IPFS to use S3 as storage layer. Using S3 as storage is expensive! IPFS is chatty and makes a LOT of S3 GET requests which are paid for.

Architecture

IPFS cluster with EFS One Zone Storage Class (Default)

IPFS Cluster With EFS One Zone Storage

IPFS cluster with Standard EFS Storage

For the region contains AZ that does not support One Zone EFS.

Set ONE_ZONE_EFS=False in ipfscluster.env

IPFS Cluster With EFS Standard Storage

Install CDK

Getting started with CDK

npm install -g aws-cdk

If it's your first time using CDK follow the instruction in the documentation to setup your environment.

Prepare CDK Environment

Make sure pythons3 and pip are installed on your computer.

Run pip install -r requirements.txt to install all the current dependencies.

The cdk.json file tells the CDK Toolkit how to execute your app.

This project is set up like a standard Python project. The initialization process also creates a virtualenv within this project, stored under the .venv directory. To create the virtualenv it assumes that there is a python3 (or python for Windows) executable in your path with access to the venv package. If for any reason the automatic creation of the virtualenv fails, you can create the virtualenv manually.

To manually create a virtualenv on MacOS and Linux:

$ python3 -m venv .venv

After the init process completes and the virtualenv is created, you can use the following step to activate your virtualenv.

$ source .venv/bin/activate

If you are a Windows platform, you would activate the virtualenv like this:

% .venv\Scripts\activate.bat

Once the virtualenv is activated, you can install the required dependencies.

$ pip install -r requirements.txt

Check the ipfscluster.env file to edit the relevent information like the username:password needed to secure the IPFS Cluster API.

At this point you can now synthesize the CloudFormation template for this code.

$ cdk synth

To add additional dependencies, for example other CDK libraries, just add them to your requirements.txt file and rerun the pip install -r requirements.txt command.

Useful commands

cdk ls list all stacks in the app
cdk synth emits the synthesized CloudFormation template
cdk deploy deploy this stack to your default AWS account/region
cdk diff compare deployed stack with current state
cdk docs open CDK documentation

Deploy the stack

Specify Stack Name

The app will retrieve the stack name from the context variable. You can reference doc for more details. The default stack name is IpfsClusterFargateStack. Following is a sample command for specifing stack name while deploy stack via CDK CLI.

cdk deploy -c stack_name=$YOUR_IPFS_CLUSTER_NAME

Or you can specify stack name in cdk.json

{
  "context":{
    ...
    "stack_name": "$YOUR_STACK_NAME",
    ...
  }
}

Configure the EFS storage

Set ONE_ZONE_EFS to True in ipfscluster.env will deploy EFS one zone file system and access poinbt on each AZ

Set ONE_ZONE_EFS to False in ipfscluster.env will deploy EFS file system cross AZs. EFS access point will be created on each AZ.

Set EFS_REMOVE_ON_DELETE to True in ipfscluster.env will DELETE the EFS file system while destroying the CDK stack. The defulat behavior is RETAIN the EFS file system.

Set ECS_EXEC to True in ipfscluster.env will ENABLE ECS EXEC Command for debugging purpose.

Prepare IPFS Cluster Parameters

For security reasons, parameters will be stored in Secret Manager and will not showed up on Cloudformation console. Only default values will show up in template. However, you need those paramters to invoke IPFS Cluster API. Take note of those parameters and keep them safe.

Download ipfs-cluster-service
Execute following command and find id and private_key in identity.json. Find secret in service.json

./ipfs-cluster-service -c /tmp/ipfs init

CDK Deploy Command

Export CDK_DEPLOY_REGION variable can specify region other than default CDK region for the deployment.

Make sure you are using the right AWS Credentials to deploy your stack in the right AWS account.

Replace $YOUR_IPFS_CLUSTER_NAME with the name you want to give to your CloudFormation stack.

export CDK_DEPLOY_REGION=ap-northeast-3; cdk deploy \
--parameters ClusterId=12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC \
--parameters ClusterSecret=221813dc706c2d1baaf0a15a8710e3c5e2072783c49d214243540044f21a7315 \
--parameters ClusterPrivateKey=CAESQGM5s/BsNF06WZ6Kzn4uAnPGsgXo6Ir3hmcW981bo8v57Jj0nPJGTB86m95dygxailhiWgVEB0qf+N8Nd7ozaxM= \
--parameters ClusterCredential=admin:p@ssw0rd
-c stack_name=$YOUR_IPFS_CLUSTER_NAME

Output

Outputs:
IpfsClusterFargateStack.IpfsClusterEndpoint = dg2xxxxxxxxxx.cloudfront.net
IpfsClusterFargateStack.IpfsGatewayEndpoint = d2xxxxxxxxxxx.cloudfront.net

List all IPFS peers

Download ipfs-cluster-ctl Get $IpfsClusterEndpoint from CDK output.

export REST_API_DNS_ENDPOINT=$IpfsClusterEndpoint
export CLUSTER_SECRET=221813dc706c2d1baaf0a15a8710e3c5e2072783c49d214243540044f21a7315
export CLUSTER_RESTAPI_BASICAUTHCREDENTIALS=admin:p@ssw0rd
./ipfs-cluster-ctl -l /dns/${REST_API_DNS_ENDPOINT}/tcp/443 \
--secret ${CLUSTER_SECRET} \
--basic-auth ${CLUSTER_RESTAPI_BASICAUTHCREDENTIALS} peers ls

Example Output:

12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC | IpfsCluster0 | Sees 2 other peers
  > Addresses:
    - /ip4/10.0.0.71/tcp/9096/p2p/12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC
    - /ip4/127.0.0.1/tcp/9096/p2p/12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC
  > IPFS: 12D3KooWRpU1e9Q7hgowMiZDjpK4xvh2jXEXRuRBud9XYGetTniK
  ...
12D3KooWH7w2NDcZWxnS885rZgaPz65NAXhv45JkMLdK1x5aJi2C | IpfsCluster1 | Sees 2 other peers
  > Addresses:
    - /ip4/10.0.2.66/tcp/9096/p2p/12D3KooWH7w2NDcZWxnS885rZgaPz65NAXhv45JkMLdK1x5aJi2C
    - /ip4/127.0.0.1/tcp/9096/p2p/12D3KooWH7w2NDcZWxnS885rZgaPz65NAXhv45JkMLdK1x5aJi2C
  > IPFS: 12D3KooWGyUcYasEGvbvPHgoGcfgxnTqT9Syg7dkW9BWWxkcbcpT
  ...
12D3KooWBhEzKxgj5CMSriTXPXtjdpkoXxuHZAJvZ77NAQqNijXW | IpfsCluster2 | Sees 2 other peers
  > Addresses:
    - /ip4/10.0.1.104/tcp/9096/p2p/12D3KooWBhEzKxgj5CMSriTXPXtjdpkoXxuHZAJvZ77NAQqNijXW
    - /ip4/127.0.0.1/tcp/9096/p2p/12D3KooWBhEzKxgj5CMSriTXPXtjdpkoXxuHZAJvZ77NAQqNijXW
  > IPFS: 12D3KooWFFRMoufVPwMoX6JiNu1wJ9yYm971kRYK16af2QLZkdsN
  ...

Add a file to the IPFS cluster

./ipfs-cluster-ctl -l /dns/${REST_API_DNS_ENDPOINT}/tcp/443 \
--secret ${CLUSTER_SECRET} \
--basic-auth ${CLUSTER_RESTAPI_BASICAUTHCREDENTIALS} add $PATH_TO_FILE

Output

added Qmc9CzkoBMoPGXt78mGcE9SAXcTnvttR8UNNXXXXXXXXXX $FILENAME

Get the IPFS file via public gateway

curl https://ipfs.io/ipfs/$CID --output $PATH_TO_SAVE

Get the IPFS file via IPFS Cluster gateway (Cloudfront Distribution)

Get $IpfsGatewayEndpoint from CDK output

curl https://$IpfsGatewayEndpoint/ipfs/$CID --output $PATH_TO_SAVE

Security

See CONTRIBUTING for more information.

License

This architecture definition and related codes are licensed under the MIT-0 License. See the LICENSE file.

aws-samples / ipfs-cluster-fargate

readme