Closed wconstab closed 3 weeks ago
i think i abandoned this becuase even with this fix the PP tracer would not work due to other issues with the fused-rmsnorm custom op registration.
did we figure out a solution there? or are we totally giving up on fused_rmsnorm and switching to compile?
cc @lessw2020
abandoned
Stack from ghstack (oldest at bottom):
301
161
The conditional expression forces evaluation of a symbolic quantity, which is not a problem in eager but prevents tracing/export and blocks using Pipeline Parallelism via tracing frontend.
Tensor.contiguous operator already handles returning
self
if the striding of self is already contiguous.Fixes https://github.com/pytorch/PiPPy/issues/1108