search

ietf-wg-cats / draft-ietf-cats-usecases-requirements

Other
0 stars 3 forks source link

Add use case of Computing-Aware AI large model #1

Closed QingAn closed 3 months ago

QingAn commented 1 year ago

Add use case of Computing-Aware AI large model, based on the discussion during IETF 117.

QingAn commented 1 year ago

It looks to me Fig 5 & 6 both are co-interence model. Does that mean, the remote cloud side always plays the key role?

During the inference, if the customized model is deployed on the edge and device is requesting the customized model's AI inference service, then the edge itself will be the key. If device is requesting the foundation model's AI inference service, then remote cloud side will be the key.

Curious to know, which is the most affecting contributor to inference, from device to edge or from edge to the cloud in terms of delay/bandwidth/computing resources? Would edge do the work alone without the help from the cloud side?

Similarly, if the customized model is deployed on the edge and device is requesting the customized model's AI inference service, then the edge can work independently.

VMatrix1900 commented 1 year ago

It looks to me Fig 5 & 6 both are co-interence model. Does that mean, the remote cloud side always plays the key role?

During the inference, if the customized model is deployed on the edge and device is requesting the customized model's AI inference service, then the edge itself will be the key. If device is requesting the foundation model's AI inference service, then remote cloud side will be the key.

Curious to know, which is the most affecting contributor to inference, from device to edge or from edge to the cloud in terms of delay/bandwidth/computing resources? Would edge do the work alone without the help from the cloud side?

Similarly, if the customized model is deployed on the edge and device is requesting the customized model's AI inference service, then the edge can work independently.

So there is a third scenario in which the edge can inference by its own without the cloud. It seems to me this scenario is the most suitable one for the CATS. Can you add a third figure to demonstrate it? And add the explanation text too.

By permutation, I wonder is there a scenario in which device and edge co-inference without the cloud?

QingAn commented 1 year ago

Thanks for this contrib.

I would shorten the "generic introductory part" but focus more on CATS-specific matters and highlight what is unique to the IA-as-an-app. Thanks.

Thanks for the comments. I have made the update and also some reply to your comment. Please take a look.

QingAn commented 1 year ago

So there is a third scenario in which the edge can inference by its own without the cloud. It seems to me this scenario is the most suitable one for the CATS. Can you add a third figure to demonstrate it? And add the explanation text too.

Done. Please take a look.

By permutation, I wonder is there a scenario in which device and edge co-inference without the cloud?

Frankly, I hadn't considered this scenario before. But theoretically, it is possible.

VMatrix1900 commented 3 months ago

Merged.