This project leverages Docker to create images that run Teevity's AWS Usage tool, Ice (formerly Netflix Ice). From Teevity's Ice repository introduction:
Ice provides a birds-eye view of our large and complex cloud landscape from a usage and cost perspective. Cloud resources are dynamically provisioned by dozens of service teams within the organization and any static snapshot of resource allocation has limited value. The ability to trend usage patterns on a global scale, yet decompose them down to a region, availability zone, or service team provides incredible flexibility. Ice allows us to quantify our AWS footprint and to make educated decisions regarding reservation purchases and reallocation of resources.
Ice is a Grails project. It consists of three parts: processor, reader and UI. Processor processes the Amazon detailed billing file into data readable by reader. Reader reads data generated by processor and renders them to UI. UI queries reader and renders interactive graphs and tables in the browser.
More information and screenshots can be found on the project's git page.
This repository seeks to ease the installation and configuration of Ice. In addition to the application container, this repository configures a nginx proxy which also helps fix URI issues I had when accessing Ice directly. After following these directions you should be able to connect to your server's IP address or FQDN over port 80 and access the Ice application. Additionally, I've supplied an Upstart job script you can leverge to start your containers on boot.
Sign up for Amazon's programmatic billing access here to receive detailed billing(hourly) reports. Verify you receive monthly billing file in the following format:
<accountid>-aws-billing-detailed-line-items-<year>-<month>.csv.zip
Create the docker-compose file: cp docker-compose-template.yml docker-compose.yml
Open docker-compose.yml and configure the Timezone as well as add the AWS Access Key ID and Secret Key that has access to the s3 billing bucket: vi docker-compose.yml
ice:
build: ice
command: |
-Djava.net.preferIPv4Stack=true
-Djava.net.preferIPv4Addresses
-Duser.timezone=<Your Timezone ie America/New_York>
-Dice.s3AccessKeyId=<s3AccessKeyId>
-Dice.s3SecretKey=<s3SecretKeyId>
Create the configuration file that will be mounted to the container: cp ice/assets/sample.properties ice/assets/ice.properties
Open ice.properties and configure a basic setup by updating the following: vi ice/assets/ice.properties
# s3 bucket name where the billing files are
ice.billing_s3bucketname=
# location for the billing bucket
ice.billing_s3bucketregion=
# Your company name
ice.companyName=
# s3 bucket name where Ice can store output files
ice.work_s3bucketname=
# Your AWS account number. You can also replace "production" with your own identifier
ice.account.production=
More information on the configurations can be found on the project's git page.
docker-compose up
This will start the containers in the forground so you can see if there are any errors.docker-compose up -d
to run the containers in the background.I've included an Upstart job in the init
directory of this repository. This will allow you to start the containers with start ice
and stop them by running stop ice
. This will also start your containers at boot.
Copy init/ice.conf
to your host's /etc/init/
directory
Edit the the job vi /etc/init/ice.conf
and change the path to the docker-compose file
pre-start exec /usr/local/bin/docker-compose -f /path/to/your/docker-compose.yml up -d
post-stop exec /usr/local/bin/docker-compose -f /path/to/your/docker-compose.yml stop
Reload the job controller initctl reload-configuration
The version of highstock.js is locked at 4.2.1 due to a breaking change in newer versions.
To adjust the memory allocation, add the following line to the docker-compose.yml
file:
...
environment:
- GRAILS_OPTS=-server -Xmx4G -Xms1G -Dfile.encoding=UTF-8
...
To launch just the Ice container via the docker run
command, issue the following:
docker run -v `pwd`/ice/assets/ice.properties:/opt/ice/src/java/ice.properties \
-e GRAILS_OPTS="-server -Xms2g -Xmx2g -server" \
jonbrouse/ice \
-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses -Duser.timezone=America/New_York -Dice.s3AccessKeyId=XXXXXXXXXXXXX -Dice.s3SecretKey=XXXXXXXXX run-app
To deploy ice to a kubernetes cluster running on AWS
Update configmap-ice.yaml ice.companyName=
, ice.billing_s3bucketname=<your-s3-detailed-billing-bucket-here>
, ice.work_s3bucketname=<your-s3-ice-work-bucket-here>
Update deployment.yaml with -Duser.timezone=<Your Timezone ie America/New_York>
,- -Dice.s3AccessKeyId=<s3AccessKeyId>
, - -Dice.s3SecretKey=<s3SecretKeyId>
Deploy kubectl apply -f ./kubernetes/deploy/
Make sure to replace the placeholders excluding the \<angle brackets>