Add multi-edge collaborative inference support

EnfangCui commented 3 years ago

Motivation

Multi-edge collaborative inference refers to the use of multiple edge computing nodes for collaborative inference. This technology can make full use of the distributed computing resources of edge computing nodes, reduce the delay of edge AI services and improve inference accuracy and throughput. It is a key technology of edge intelligence Therefore, we propose a multi-edge collaborative inference framework to help users build multi-edge collaborative AI business easily based on KubeEdge.

Goals

The framework can utilize multiple edge computing nodes for collaborative inference.
Utilize KubeEdge's EdgeMesh to realize multi-edge load balancing.
Provide typical multi-edge inference case study (such as ReID, multi-source data fusion, etc.).

Solution

Take pedestrian ReID as a example:

pedestrian ReID workflow

The client is used to read the camera video data and carry out the first stage of inference. The server is used to receive the region proposal predicted by the client for final target detection and pedestrian feature matching.

CRD example (preliminary design)

apiVersion: sedna.io/v1alpha1
kind: MultiEdgeInferenceService
metadata:
name: pedestrian-reid
namespace: default
spec:
clientWorkers:
-template:
  spec:
    containers:
      - image: kubeedge/sedna-example-multi-edge-inference-reid-client:v0.1.0
      ......
    nodeSelector:
      ......
serverWorkers:
-template:
  metadata:
  labels:
    app: reid-server
  spec:
    containers:
      - image: kubeedge/sedna-example-multi-edge-inference-reid-server:v0.1.0
      ......
    nodeSelector:
      ......
-template:
  metadata:
  labels:
    app: user-server
  spec:
    containers:
      - image: kubeedge/sedna-example-multi-edge-inference-user-server:v0.1.0
      ......
    nodeSelector:
      ......
-template:
  metadata:
  labels:
    app: mqtt-server
  spec:
    containers:
      - image: kubeedge/sedna-example-multi-edge-inference-mqtt-server:v0.1.0
      ......
    nodeSelector:
      ......

Open issues

We hope to build a general framework to adapt to a variety of multi-edge collaborative inference scenarios. Therefore, users and developers are welcome to put forward more scenarios, applications and requirements to improve our architecture and CRD interface design.

TymonXie commented 3 years ago

Hi, Enfang, i think the concepts of 'client' and 'server' is too extensive, services and modules can not be distinguished.

TymonXie commented 3 years ago

in my opinion, because we need a edge-edge collaborative inference, the definition of fully distributed may be more appropriate. for example, we can replace 'clientWorkers' and 'serverWorkers' with 'inferenceWokers', and we can add a key words, such as 'collaborative algorithm‘，etc.

ugvddm commented 3 years ago

It will be better to summit a proposal, we can comment it. right?

EnfangCui commented 3 years ago

It will be better to summit a proposal, we can comment it. right?

Ok, I will submit a proposal tomorrow.

llhuii commented 3 years ago

/close I close this due to this work has been transfered to the work of object search and tracking, see this pr #100 for more detail.

Feel free to reopen it.

kubeedge-bot commented 3 years ago

@llhuii: Closing this issue.

In response to [this](https://github.com/kubeedge/sedna/issues/77#issuecomment-897323279): >/close >I close this due to this work has been transfered to the work of object search and tracking, see this pr #100 for more detail. > >Feel free to reopen it. Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

kubeedge / sedna