search

datacratic / StarCluster

StarCluster is a utility for creating and managing computing clusters hosted on Amazon's Elastic Compute Cloud (EC2).
http://star.mit.edu/cluster
GNU Lesser General Public License v3.0
37 stars 13 forks source link

Modify AMI block device mapping on instance start #63

Open johncolby opened 8 years ago

johncolby commented 8 years ago

I am using an EBS backed AMI that doesn't by default make instance store/ephemeral space available. Is there a way via starcluster to modify the block device mapping on instance startup? As described in http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html#Using_OverridingAMIBDM I'd rather not have to make a whole new snapshot just for this...

Thanks, John

FinchPowers commented 8 years ago

Do you mean this https://github.com/datacratic/StarCluster/blob/55e4ca086bdc0fff06b89d658edce6ba3df16881/starcluster/templates/config.py#L183 ?

johncolby commented 8 years ago

No...but maybe I can fake it in via that EBS config? I want to have it add an instance store/ephemeral device on AMI startup that I can use for /scratch. The AMI I am using is HVM-EBS, so when starcluster starts it up by default there is no ephemeral storage accessible.

root@nitrcce:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       99G   36G   58G  39% /
udev            7.9G  8.0K  7.9G   1% /dev
tmpfs           1.6G  724K  1.6G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.8G  144K  7.8G   1% /run/shm
root@nitrcce:~# mount
/dev/xvda1 on / type ext4 (rw)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
gvfs-fuse-daemon on /var/lib/lightdm/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=lightdm)

root@nitrcce:~# fdisk -l

Disk /dev/xvda: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders, total 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *       16065   209712509   104848222+  83  Linux