search

webmachinelearning / meetings

🗒 Agendas, minutes and other meeting material
Other
18 stars 4 forks source link

Collect use cases #1

Closed tomoyukilabs closed 5 years ago

tomoyukilabs commented 5 years ago

This issue is to collect low- and high-level use cases of WebML API to input to the F2F meeting at TPAC 2018. https://github.com/webmachinelearning/meetings/blob/master/2018-10-26-lyon/README.md#1600-target-demographic-use-cases-and-requirements-all

tomoyukilabs commented 5 years ago

I would like to input the following use cases at application and API levels. I'd be pleased if those could be useful for WebML CG. Any comments are welcome.

High-Level Use Cases

This section illustrates application-level use cases for the Web Machine Learning API (WebML API). All applications in those use cases can be built on top of pre-trained deep neural network (DNN) models.

Person Detection

A user is browsing a social network website and wishes to take a photo and upload it to the website. Before the photo is uploaded, the website runs SSD (Single Shot MultiBox Detector) or YOLO on the WebML API to detect regions that include persons so that the user can filter and de-personalize irrelevant persons on it.

Skeleton Detecton

A user opens a web application that continuously captures her body with her smartphone's camera. The web application extracts her skeleton by running PoseNet on the WebML API to recognize her gesture or body language. When she strikes a specified pose like raising a hand, the web application automatically takes a photo and upload it to an online storage.

Random Image Generation

A user wishes to make her new account and looks for a new icon image. When she clicks a "Generate" button on the webpage for creating an account, the webpage runs a generator model of generative adversarial network (GAN) for icon synthesis on the WebML API. She can repeat random icon generation until she finds her favorite one.

Low-Level Use Cases

This section collects API-level use cases for the WebML API. My assumption is that ML frameworks would refer to WebML API so that application developers could make use of those capabilities via the frameworks.

Custom Layer

A web application developer wants to run a DNN model on the WebML. However, she has found that some of activation functions like Leaky ReLU, ELU, etc. are not included in the WebML API. So she builds custom layers of the additional activation functions on top of the WebML API. Note that the scope of custom layers may include convolution, normalization, etc. as well as activation.

Network Concatenation

A web application developer is trying to implement a DNN model that refers to outputs from some convolutional layer in MobileNets or ResNet as a feature map. She wants to use a pre-trained model of these networks to reduce training time. So she implements her network architecture so that its input is the output of convolutional layers of pre-trained MobileNets or ResNet models.

Performance Adaptation

A web application developer has a concern about performance of her DNN model on mobile devices. She has confirmed that the model runs too slow on mobile devices which does not have GPU acceleration. So her web application refers to the WebML API to confirm whether acceleration is available or not, so that the application can display the warning for devices without acceleration.

After several weeks, she has developed a tiny DNN model that can even run on CPU. So she modifies the application so that the application loads the tiny model in the case of CPU-only devices.

anssiko commented 5 years ago

Thanks! Added to the F2F agenda.

(All - let’s use this issue to document any other use cases.)

cynthia commented 5 years ago

I believe the above text should s/adversarial generative network/generative adversarial network/

tomoyukilabs commented 5 years ago

@cynthia Correct. Thanks!

anssiko commented 5 years ago

@tomoyukilabs, at F2F, we agreed to add use cases directly to the specification to make it obvious they inform the spec work. There was consensus that your use cases could be used as a starting point to be iterated further based on F2F feedback.

To get us started with this process, feel free to submit a PR against the newly created spec repo https://github.com/webmachinelearning/webnn to add your use cases to the skeleton spec. There's an empty "Use cases" section at https://webmachinelearning.github.io/webnn/#usecases ready for your contributions.

The initial contribution does not need to be perfect since this section is expected to evolve until the group feels it's ready to advance to the solution phase aka API specification drafting.

Thank you!

tomoyukilabs commented 5 years ago

@anssiko Okay, I'll submit a PR to add use cases (maybe in next week). Thanks!

anssiko commented 5 years ago

@tomoyukilabs, I forgot to mention the use cases PR will go through the usual PR review flow to capture suggestions and contributions from F2F discussion so you can basically make the initial commit based on what you have in https://github.com/webmachinelearning/meetings/issues/1#issuecomment-430846472 and we'll iterate on it as a group before merging. This way we'll also get a better paper trail of changes attributed to the right people.

tomoyukilabs commented 5 years ago

@anssiko I have submitted a PR https://github.com/webmachinelearning/webnn/pull/2 that includes use cases copied from https://github.com/webmachinelearning/meetings/issues/1#issuecomment-430846472. When a review to capture the F2F discussion is posted, I'll follow the review by updating the PR.

anssiko commented 5 years ago

This issue was addressed in PR https://github.com/webmachinelearning/webnn/issues/2