search

kubeedge / sedna

AI tookit over KubeEdge
https://sedna.readthedocs.io
Apache License 2.0
505 stars 163 forks source link

Edge AI Benchmark: Consolidate/Prioritize metrics #118

Open YuxiaoHu opened 3 years ago

YuxiaoHu commented 3 years ago

The draft benchmark version looks good, which is a nice start point. Thanks for the great efforts!

Issues: There are too many metrics, which lead to two problems:

  • Difficult for readers to understand each of them according to the short descriptions. This proposal should be self-contained, so that the readers won’t need to read all the references. It will be nice to add descriptions/examples/Figures for each of the metrics;
  • For new users, it will be difficult to execute. I suggest we start from one or two core metrics for each scenario (like P0 features), and provide additional metrics (as P1/P2, …) for advanced users or readers who are interested.
MooreZheng commented 2 years ago

Thanks. We have updated the first draft of the specification.

The specification is still open to edit. Any interested member please also refer to https://github.com/kubeedge/sedna/issues/96

MooreZheng commented 2 years ago

Add as suggested, e.g., https://github.com/kubeedge/ianvs/tree/main/examples/pcb-aoi/singletask_learning_bench/testenv