TankLabTJU / INFless

The source code of INFless,a native serverless platform for AI inference.
GNU General Public License v3.0
33 stars 14 forks source link

A little confusion about one image of the paper #3

Open penghuima opened 1 year ago

penghuima commented 1 year ago

The author's work is great!There's a question i want to ask: context:

cf96628da6f872b67495d6222f7e9d2` Figure 3(a) shows the number of instances activated for serving ResNet-20 with and without OTP batching. By aggregating the user requests into batches (we set 𝑏𝑎𝑡𝑐ℎ𝑠𝑖𝑧𝑒 = 4), the total number of function invocations declines by 72%. As an instance may be reused by a later-arriving request,the total number oflaunched instances under batching also declines by 35%.

Does w/o batching mean there is no batch processing? If the orange dotted line represents multiple concurrent request processing for a single instance, shouldn't the number of instances represented by the orange dotted line be less than the number of instances represented by the blue dotted line.

ynyang1 commented 1 year ago

Thanks for your question, we have found a mistake in this picture. The orange line shoule be w/o batching and the blue one shoule be OTP batching.