define cfncluster configuration

[kdaily]

How do I run this with an existing cfncluster configuration?

[kdaily]

The dependencies identified in #2 should be added using pre- or post-install scripts (https://cfncluster.readthedocs.io/en/latest/pre_post_install.html)

[bintriz]

[aws]
aws_region_name = us-east-1
aws_access_key_id = xxxx
aws_secret_access_key = xxxx

[cluster mycluster]
vpc_settings = myvpc
ebs_settings = myebs
key_name = tj-macbook
ec2_iam_role = CfnClusterInstanceRole
post_install = s3://abyzovlab/cfncluster/post_install.sh
master_instance_type = c5.large
#compute_instance_type = c5.9xlarge
compute_instance_type = m5.12xlarge
initial_queue_size = 0
max_queue_size = 1000
cluster_type = spot
#spot_price = 1.0
spot_price = 2.0

[vpc myvpc]
master_subnet_id = subnet-xxxx
vpc_id = vpc-xxxx                          # for EFS mount
additional_sg = sg-xxxx                    # for EFS mount

[ebs myebs]
ebs_snapshot_id = snap-xxxx       # backup snapshot for /shared

[global]
update_check = true
sanity_check = true
cluster_template = mycluster

This is our current cfncluster configuration. I changed some ids or credential info specific to our group into "xxxx".

[bintriz]

#!/bin/bash

# EFS
mkdir -p /efs
echo "fs-xxxxxx.efs.us-east-1.amazonaws.com:/ /efs nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 0 0" >> /etc/fstab
mount -a -t nfs4

# Additional packages
/shared/apps/cluster_setup/install_packages.sh

# Timezone
sed -i '/ZONE/s/UTC/America\/Chicago/' /etc/sysconfig/clock
ln -sf /usr/share/zoneinfo/America/Chicago /etc/localtime
reboot

This is the current post_install.sh script. What this does is 1) mounting EFS, 2) running package install script, and 3) setting time zone.

[bintriz]

For NGS tools and dependencies, current implementation is not automatic. I just installed all dependencies manually into /shared. Then, /shared was backed up as a EBS snapshot. If I need to recreate a cfncluster, this cfncluster configuration restores /shared from there.

[kdaily]

@bintriz can you please provide install_packages.sh, even if it is partially manual?

[kdaily]

@bintriz I cannot move forward with the first phase of our plan until I have the install_packages.sh script.

[bintriz]

Yes, I'm currently writing and testing it step by step. Soon, I will upload it.

[kdaily]

13 closes this.