Closed RaulPPelaez closed 1 year ago
Yes!
should we describe in the README something about enabling the option of using/not using the optimizations?
What option are you referring to?
The static shapes thing for example, which I am seeing defaults to True in TensorNet, and I understand this needs to be done to use CUDA graphs. But don't take very seriously my thoughts, I might be wrong, you decide.
Yeah... there is no option to tune that from outside, its always true. There is no other performance-only option, so I did not want to include it just for that. Additionally, for that particular option, there is really no good heuristic AFAIK. I tried to enable it only if cuda graph capture is enabled but torch really dislikes these kind of model behavior change and I run into issues. So I just left it as True.
At some point it would be interesting to add a set of performance-only options, but I think this is not the time.
is this done Raul?