search

kubeflow / katib

Automated Machine Learning on Kubernetes
https://www.kubeflow.org/docs/components/katib
Apache License 2.0
1.48k stars 439 forks source link

[discussion] Support Advanced NAS Algorithms #461

Open gaocegege opened 5 years ago

gaocegege commented 5 years ago

Now we are using a custom IR (architecture + nn_config) to represent a candidate model, which may be insufficiently expressive, I think.

I think we should use mature IRs such as Relay IR or ONNX to represent the candidate model.

/cc @DeeperMind @andreyvelich

gaocegege commented 5 years ago

Ref https://github.com/dmlc/tvm/issues/1673

andreyvelich commented 5 years ago

@gaocegege This is make sense. What changes in API do we need to make IR like Relay IR or ONNX. And what will be the best for the NAS tasks? /cc @johnugeorge @richardsliu @hougangliu @YujiOshima

gaocegege commented 5 years ago

We need to investigate which IR works well for the case. microsoft/nni uses ONNX. While I think ONNX is not expressive.

@anchovYu is working on this issue.

andreyvelich commented 5 years ago

That is cool! Just let us know which structure you want to use and will our definition of Neural Network work for that. For RL you can find it here.

gaocegege commented 5 years ago

@andreyvelich Yeah, sure. Maybe we could discuss it in the katib meeting. Actually, we are not sure about the choice, too.

There are several ways to represent the search space of NAS: chain-structured (what we are using now), chain-structured with jumps and other advanced links, and cell-structured. I cannot figure a way to represent such complex space definition, even via ONNX or Relay IR or other IRs.

andreyvelich commented 5 years ago

Alright, let discuss at Katib meeting in this week. /cc @DeeperMind

gaocegege commented 4 years ago

https://github.com/microsoft/onnxruntime/tree/master/orttraining ONNX runtime supports training ONNX. Maybe we could have a look.

/cc @andreyvelich

issue-label-bot[bot] commented 4 years ago

Issue-Label Bot is automatically applying the labels:

Label Probability
area/katib 0.74
kind/question 0.84

Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback! Links: app homepage, dashboard and code for this bot.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

andreyvelich commented 3 years ago

/lifecycle frozen