search

microsoft / nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
https://nni.readthedocs.io
MIT License
13.88k stars 1.81k forks source link

Case study suggestions for making Retiarii accessible #3873

Open sayakpaul opened 3 years ago

sayakpaul commented 3 years ago

What would you like to be added:

More case studies on the byproducts of Neural Architecture Search:

These case studies do not need to fully implemented. Minimal demonstrations should be sufficient enough for users to expand the case studies.

Also being able to specify hardware-aware constraints in the search space is an important use-case for NAS. An example showing how to do that would be useful.

Why is this needed:

Without establishing NNI's Retiarii capabilities transfer well to the existing line of works, it might be hard for researchers to adapt Retiarii for their own use.

QuanluZhang commented 3 years ago

@sayakpaul , thanks for your great suggestions. we have an WiP pr here which implemented the space of Swish. We expect users to directly use this module in their PyTorch/TensorFlow model to automatically find out the best activation function specifically for their model.

sayakpaul commented 3 years ago

Great. I will look into it for sure. Should I keep this issue open for other suggestions?

QuanluZhang commented 3 years ago

Great. I will look into it for sure. Should I keep this issue open for other suggestions?

yes, we can keep it open for now :)

sayakpaul commented 3 years ago

Sure, thank you. You might want to add a relevant label to this issue for better/easier triage.

ultmaster commented 3 years ago

Added your vote for MobileNetV3 and MobileDet. Will consider them in future planning. :)

sayakpaul commented 3 years ago

Sure. I personally think teaching the developers how to go about constructing informative search spaces itself is very rewarding.