search

mdibl / biocore_utils

Resources for RNA-SEQ pipeline analysis
0 stars 2 forks source link

Setup Jenkins server #2

Open LucyHut opened 6 years ago

LucyHut commented 6 years ago

Steps to setup MDIBL's Jenkins master with On-demand and Spot fleet slaves

See: 1) Setup Jenkins Server 2) Launch an Elastic Beanstalk Environment 3) Install s3fs file system software on your Ec2 instance

LucyHut commented 6 years ago

Amazon EC2 Console

1) Create EC2 instance -> Image -> launch configuration -> launch template -> Spot fleet 2) Create cloud user -> access Key ID and Secret key

* Create a new user group "jenkins" (IAM -> Groups)
* Set the following policies for the new group
   a) AmazonS3FullAcess
   b) AmazonEC2FullAccess
* Create a new user "ec2-user" with programtic access(IAM->Users -> Add user)
   a) Add user to "jenkins" group
   b) Create Security credentials (IAM -> ec2-user->security credentials ->Create access Key)
      since the secret key is generated only once, it's advisible to copy
      both the Access Key ID and the security key  and save them in a file somewhere
      in your system where only you can read it
LucyHut commented 6 years ago

Install AWS plugins on Jenkins Server

Manage Jenkins -> Manage plugins

a) install Amazon EC2 plugin
b) install EC2 Fleet plugin
LucyHut commented 6 years ago

Add A Spot Fleet

Manage Jenkins -> Configure System -> Cloud -> Add New Cloud -> Amazon SpotFleet

    a) AWS Crendentials -> Add -> Jenkins Credentials
       Kind: AWS Creadentials
       Access Key ID : set to access key ID created earlier
       Secret Access Key: ste to security key generated ealier

    b) Region : select the region of the fleet created earlier
    c) Spot Fleet: select the availbale fleet
    d) Launcher: select 'Launch agent agents via SSH'
    e) Credentials: select the cloud user with credentials used
LucyHut commented 5 years ago

Add Ec2 instance provisions

Manage Jenkins -> Configure System -> Cloud -> Add New Cloud -> Amazon EC2

     a) AWS Crendentials -> Add -> Jenkins Credentials
       Kind: AWS Creadentials
       Access Key ID : set to access key ID created earlier
       Secret Access Key: ste to security key generated ealier

    b) Region : select the region of the fleet created earlier
    c) EC2 Key Pair's Private Key -- copy and paste the keypair used to connect to your EC2 instances
    d) AMI ID -> copy and paste the AMI ID to use for aws console
    e) select the Instance Type
    f) Security group names -> copy and paste the security group name from aws console
    g) Set Remote FS root to /home/ec2-user/jnkins
    h) set Remote user to ec2-user
LucyHut commented 5 years ago

Note