FIRM: An Intelligent Fine-grained Resource Management Framework for SLO-Oriented Microservices

[pentium3]

https://www.usenix.org/conference/osdi20/presentation/qiu

[pentium3]

goal

FIRM\cite{FIRM} designs an intelligent adaptive resource management framework for optimizing microservices latency SLO using online telemetry data and machine-learning methods.

solution

1. Tracing Coordinator [sec3.1]

• collect span for each microservice: unit time processing 1 request
  • also collect system metrics like cpu/mem/network/disk utilization for each microservice instance (see Table 2)
• construct execution history graph
• profile and update graph dynamically over time

2. calculate critical path in the execution history graph [sec3.2]

a weighted longest path algorithm. see Algorithm 1

3. detect critical microservice instances (which most likely to cause request SLO violation) [sec3.3]

• collect a series of latency (latency of each request on each microservice) data over time.
• For each microservice $i$ in the critical path $CP$, calculate
  • $PCC(Ti, T{CP})$ : how likely the variance of microservice $i$ contributes to the variance of $T_{CP}$ (variance of e2e latency of the request)
  • $T{99}/T{50}$ : higher value means higher congestion in the queue. lower value means most of the requests are handled normally
  • use SVM. call $SVM(PCC(Ti, T{CP}), T{99}/T{50})$ to classify whether microservice $i$ is a critical component (contributes significantly to SLO violation)

4. RL

RL basic:

FIRM utilizes the deep deterministic policy gradient (DDPG) algorithm, which is a model-free, actor-critic RL framework (shown in Fig. 7).

• state $s_t$ : performance & resource measurements at time $t$ for a critical service instance (see Table 3)
• action $a_t$ : decide Resource Limits at time $t$ for a critical service instance (see Table 3)
• policy $\pi_\theta$ : a policy is a strategy or a rule that the agent follows to return an action for a given state. The ultimate goal is to find an optimal policy π* that maximizes the expected cumulative reward over time.
• reward value $V_t$ : expected reward if we apply a policy π under a state $s_t$.
• critic : estimates the reward value (i.e., the expected value of cumulative discounted function reward under a given policy)
• actor : updates the policy in the direction suggested by the critic.
• Model-free: learn optimal behavior directly from interactions with the environment, without explicitly modeling or learning the environment's dynamics

how does FIRM apply RL:

• The actor-critic framework combines policy-based and value-based methods (i.e., consisting of an actor-net and a critic-net as shown in Fig. 8), and that is suitable for continuous stochastic environments, converges faster, and has lower variance
  • actor net : The Actor Net represents the policy π, which maps a state to an action. It decides how to adjust resource limits based on the current state of the microservice environment.
  • critic net : The Critic Net estimates the action-value function $Q_w(s_t, a_t)$ . It estimates the expected cumulative reward for taking a specific resource allocation action in a given system state.
    • here $r_t = \alpha \cdot SMt \cdot | \mathcal{R} | + ( 1 −\alpha ) \cdot \sum{i}^{| \mathcal{R} |} RU_i / RLT_i$ is the reward function. $\mathcal{R}$ is the set of resources.
    • It measures the immediate benefit of the action taken, considering both SLO maintenance and resource utilization.
    • The goal of the RL agent is, given a time duration t, to determine an optimal policy $\pi$ that results in as few SLO violations as possible (i.e., minpt SMt ) while keeping the resource utilization/ limit as high as possible (i.e., maxpt RUt/RLTt ).

• resource model: Each microservice instance is deployed in a separate container with a tuple of resource limits $RLT=(RLT{cpu}, RLT{mem}, RLT{llc}, RLT{io}, RLT_{net})$
  • consider CPU utilization, memory bandwidth, LLC capacity, disk I/O bandwidth
• At each discrete time step t, the agent observes a state of the environment $s_t \in S$ , and performs an action $at \in A$ based on its policy $\pi\theta (s)$ ( parameterized by $\theta$ ), which maps state space $S$ action space $A$.