Open gaocegege opened 5 years ago
@gaocegege This is make sense. What changes in API do we need to make IR like Relay IR or ONNX. And what will be the best for the NAS tasks? /cc @johnugeorge @richardsliu @hougangliu @YujiOshima
We need to investigate which IR works well for the case. microsoft/nni uses ONNX. While I think ONNX is not expressive.
@anchovYu is working on this issue.
That is cool! Just let us know which structure you want to use and will our definition of Neural Network work for that. For RL you can find it here.
@andreyvelich Yeah, sure. Maybe we could discuss it in the katib meeting. Actually, we are not sure about the choice, too.
There are several ways to represent the search space of NAS: chain-structured (what we are using now), chain-structured with jumps and other advanced links, and cell-structured. I cannot figure a way to represent such complex space definition, even via ONNX or Relay IR or other IRs.
Alright, let discuss at Katib meeting in this week. /cc @DeeperMind
https://github.com/microsoft/onnxruntime/tree/master/orttraining ONNX runtime supports training ONNX. Maybe we could have a look.
/cc @andreyvelich
Issue-Label Bot is automatically applying the labels:
Label | Probability |
---|---|
area/katib | 0.74 |
kind/question | 0.84 |
Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback! Links: app homepage, dashboard and code for this bot.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
/lifecycle frozen
Now we are using a custom IR (architecture + nn_config) to represent a candidate model, which may be insufficiently expressive, I think.
I think we should use mature IRs such as Relay IR or ONNX to represent the candidate model.
/cc @DeeperMind @andreyvelich