ONNXNaiveNASflux tries to merge certain op-types into a single vertex. The most prominent example of this is how activation functions are separate nodes in ONNX, but Flux allows them as part of many layer types and fuses the broadcast of the activation and bias addition.
It would be useful to be able to enable/disable this behaviour from the loading API.
One that probably should not be enabled by default is the fusing of Squeeze and Global pooling layers since Flux native global pooling layers do not squeeze the singleton dimensions (current implementation is from before Flux shipped a global pool). The merging is quite useful when doing NAS stuff since one probably don't want to randomly insert operations between the global pool and the squeeze, but I guess most users are not using this library for NAS stuff (and if they do, they would be able to easily re-enable it when loading).
ONNXNaiveNASflux tries to merge certain op-types into a single vertex. The most prominent example of this is how activation functions are separate nodes in ONNX, but Flux allows them as part of many layer types and fuses the broadcast of the activation and bias addition.
It would be useful to be able to enable/disable this behaviour from the loading API.
One that probably should not be enabled by default is the fusing of Squeeze and Global pooling layers since Flux native global pooling layers do not squeeze the singleton dimensions (current implementation is from before Flux shipped a global pool). The merging is quite useful when doing NAS stuff since one probably don't want to randomly insert operations between the global pool and the squeeze, but I guess most users are not using this library for NAS stuff (and if they do, they would be able to easily re-enable it when loading).