search

pytorch / glow

Compiler for Neural Network hardware accelerators
Apache License 2.0
3.21k stars 688 forks source link

HostManager not calling optimize on its Functions? #2694

Closed bertmaher closed 5 years ago

bertmaher commented 5 years ago

In experimenting with the HostManager I found crashes with the Habana backend due to not running the optimizer. I'm not sure what's going on here because HostManager::addNetwork does, in fact call optimizeFunction.

PRs #2689 and #2693 are workarounds for this, but I'd rather not land them since they're duplicating functionality that we should already have in HM.

jfix71 commented 5 years ago

Some of this functionality currently lives in ExecutionEngine::compile(). It might make sense to share the logic both there and in the HostManager.

nickgg commented 5 years ago

Isn't this as simple as adding a call to optimizeFunction inside addNetwork?

jfix71 commented 5 years ago

At least for #2693, checking isOpSupported() is only done inside ExecutionEngine::compile(), not optimizeFunction(). I'm not sure what was going on with #2689, as @bertmaher noted HostManager::addNetwork() already calls optimizeFunction(), so not sure what the problem is there.

nadavrot commented 5 years ago

@jfix71 @bertmaher Can we close this PR?

jfix71 commented 5 years ago

2800 partially addresses this by checking isOpSupported() at the end of optimizeFunction(). Still not sure what the issue was where @bertmaher was seeing unoptimized functions somehow.

bertmaher commented 5 years ago

I still need to check whether this is working properly with the open source repo now. I landed a temporary fix (TM) locally and haven't revisited it in a while.

bertmaher commented 5 years ago

@arunm-git tested this and we're good without the optimize call in H backend. I love it when bugs are solved automatically :)