The author's work is great!There's a question i want to ask:
context:
`
Figure 3(a) shows the number of instances activated for serving ResNet-20 with and without OTP batching. By aggregating the user requests into batches (we set 𝑏𝑎𝑡𝑐ℎ𝑠𝑖𝑧𝑒 = 4), the total number of function invocations declines by 72%. As an instance may be reused by a later-arriving request,the total number oflaunched instances under batching also declines by 35%.
Does w/o batching mean there is no batch processing? If the orange dotted line represents multiple concurrent request processing for a single instance, shouldn't the number of instances represented by the orange dotted line be less than the number of instances represented by the blue dotted line.
The author's work is great!There's a question i want to ask: context:
Does w/o batching mean there is no batch processing? If the orange dotted line represents multiple concurrent request processing for a single instance, shouldn't the number of instances represented by the orange dotted line be less than the number of instances represented by the blue dotted line.