search

kubeedge / sedna

AI tookit over KubeEdge
https://sedna.readthedocs.io
Apache License 2.0
497 stars 160 forks source link

Edge AI Benchmark review #96

Open jaypume opened 3 years ago

jaypume commented 3 years ago

Edge AI Benchmark Draft pdf is here。

If you have comments about this proposal, you can directly reply under this issue. If you want to join the benchmark writing, you can contact Dr. Zimu Zheng.

Dr. Zimu Zheng Email: zimu.zheng@hotmail.com WeChat ID: moodzheng

MooreZheng commented 3 years ago

We are still calling for:

  1. Core scenarios;
  2. Learning scheme and related benchmark survey;
  3. Test environments and settings.
MooreZheng commented 3 years ago

If anyone would like to be a reviewer of the benchmark, please contact me for registeration by email zimu.zheng@hotmail.com. A registered reviewer can have direct access to the up-to-date overleaf document, read overleaf reviews and have more timely response on submitted issues.

For a registered reviewer, the suggested review procedures on Overleaf are:

  1. [Read current issue] To avoid duplicated or conflicts comments, before submit comments, reviewers are highly suggested to read current open issues and live reviews about the benchmark on Github https://github.com/kubeedge/sedna/issues and also on Overleaf: image image

  2. [Put comments on Overleaf] When a reviewer want to add comment on a particular sentence in the benchmark, first select the sentence and click add comment on top right. image

  3. [Create issues on Github] If there are critical comments that a reviewer would to discuss, and keep track of related tasks, enhancements, and bugs for the benchmark, the reviewer can comment in this issue or create new issues in https://github.com/kubeedge/sedna/issues image

MooreZheng commented 3 years ago

If anyone would like to be an author of the benchmark, please contact me for registeration by email zimu.zheng@hotmail.com. A registered author can have direct access to the up-to-date overleaf document, have the right to write or modify benchmark content, read overleaf reviews and have more timely response on submitted issues.

For a registered author, the suggested writing procedures on Overleaf are:

  1. [Read current comments] To avoid duplicated or conflicts comments, before submit comments, an author is highly suggested to read current open issues and live reviews about the benchmark on github https://github.com/kubeedge/sedna/issues and also on overleaf: image image

  2. [Start writing] By double-clicking on the corresponding position on the PDF, Overleaf will bring the author to the specific position. image image

  3. [Recompile on Overleaf] To see whether the writing is appeared as expected, click recompile to see the results. image

  4. [Reply to the comments] Let other members know your work. Reply to the corresponding comment on Overleaf and issue on Github. If there are no comments on the writing issue, an author is highly suggested to write one at least on Overleaf, e.g., "Lack of explanation of the abc metric. Have written correspondingly". We can go over all member's work on the community meeting.
    image image image

  5. [Commit to Github] Click menu -> sync: github. An author is highly suggested to push the code with the contribution together with the Github issue link. For example, the contribution can be written like "Add explanation to Metric ABC (https://github.com/kubeedge/sedna/issues/xxx)". image image image image image

MooreZheng commented 3 years ago

Related Open Issues from Review: #118 Edge AI Benchmark: Consolidate/Prioritize metrics #119 Edge AI Benchmark: add step by step instructions for users to benchmark their works #120 Edge AI Benchmark: add leaderboard/forum