boxidau / rax-autoscaler

Apache License 2.0
5 stars 2 forks source link

Explanation of documentation, specifically config.json #87

Closed jlowe64 closed 9 years ago

jlowe64 commented 9 years ago

Hello. I am attempting to migrate from a system developed internal to Rackspace to this in test. I am having a bit of trouble finding out exactly where to get some of this information. I have also been keeping notes of things that I have ran into while installing this with very basic understanding of interacting with the Rackspace API. So some patience would be appreciated.

Scale Up Policy UUID - is this the number within the webhook URL? Where is this located? Scale Down Policy UUID - same as above. Check Type (agent.cpu, agent.load_average...) - Is RAM also an option right now? I saw some reference to this in waffle. Metric Name (depends on the check type) - Not sure what options should even be here. Webhooks Url (Pre & Post commit url(s) for scale up/down) - Can I get an explanation of "Pre", "Post" and why a Webhook URL is necessary when the UUID was provided? Also, will I be needing 4 webhooks in total? 2 for up and 2 for down?

Thank you!

jon-walton commented 9 years ago

Hi,

There are a few things to configure both on the Rackspace API side (essentially creating the autoscale group and setting the launch configuration) and on our side.

First up, create the autoscaling group in the portal and create 2 policies, 1 to scale up and another to scale down. The policy scale trigger should be set to webhook url.

The things we need now on the Rackspace side aren't available in the portal so you'll have to make use of https://cloud-api.info/ to quickly interact with the API. To login, you'll need your API key which is available from the portal under the "Account Settings" screen.

I'm going to quote the config/config-template.json file to make it easier to explain

  "autoscale_groups": {
      "group0": {

"group0" is just a name used internally by autoscale.py to make it easier for the user to reference the scaling group. It can stay as group0 or be changed to match what you've called the group in the portal

          "group_id": "group id",

"group_id" is the UUID of the scaling group, this can be seen in the portal or in pitchfork by calling the "List Scaling Groups" API method https://cloud-api.info/autoscale/#list_scaling_groups-autoscale

          "scale_up_policy": "scale up policy id",
          "scale_down_policy": "scale down policy id",

scale_up_policy and scale_down_policy requires the UUID of the policies configured as webhooks in the portal. You can retrieve the UUID in pitchfork by calling the Get Policies List method https://cloud-api.info/autoscale/#get_policies_list-autoscale

          "check_type": "agent.load_average",
          "check_config": {},
          "metric_name": "1m",

currently, the only check type we have is load_average (ram utilisation is on the backlog)

Webhook urls are an optional URL to call when we scale up or down. Pre is called before we call the Rackspace API, post is called after. You can configure multiple urls (or just a single url) to call at each stage.

siso commented 9 years ago

http://rax-autoscaler.readthedocs.org/en/latest/QuickStart.html#config-json

jlowe64 commented 9 years ago

That's a great explanation of those particular areas.

Last question. The pre/post webhooks. Currently I have two webhooks, one for up and one for down. I have configured the scale up policy and scale down policy accordingly. With webhooks, what is the point of having a pre and post? Since I only have two, would I just put those URLs in that area or would I create other webhooks?

Thank you!

jon-walton commented 9 years ago

I didn't explain the webhooks very well so hopefully I can expand on my previous post. So there are 2 places webhooks are mentioned.

The first area you'll see them mentioned is in the rackspace portal.

In the portal, there are 4 types of scaling policy

  1. Schedule once
  2. Schedule repeatedly
  3. Schedule using cron syntax
  4. Webhook

Because rax-autoscale wants to control when a policy is executed, the policy in the rackspace portal should be configured as a webhook as this is the only type of policy that won't execute automatically.

Rax-autoscale doesn't actually use the webhook URL provided by the portal so that can be ignored.

The second area you'll see webhooks mentioned is in the rax-autoscale config

The pre and post URL in the rax-autoscale config is useful if you want your application to be notified of a scaling event. You can configure a specific URL in your application to receive a POST request when we scale

The flow can essentially be boiled down to

  1. Rax-autoscale decides to scale up or down
  2. All URLs configured in the relevant policies webhook section "pre" are sent a POST request
  3. Rax-autoscale calls the rackspace api to execute a scaling policy
  4. All URLs configured in the relevant policies webhook section "post" are sent a POST request

If you put the webhook url provided by the rackspace portal in the webhook configuration section, you may find that you scale up or down twice unexpectedly. This is due to us executing the scaling policy via the api and again via the webhook

jlowe64 commented 9 years ago

That is a perfect explanation! Thank you very much.

siso commented 9 years ago

Thank you @jlowe64 for asking. Please, let us know how you progress with rax-autoscaler.