Naive addition of MPS support for the transformers service.
At the moment the service crashes python when load is high, still looking into what qualifies as high but it seems like pytorch is crashing python due to some MPS issue.
The issue manifests as some embeddings are vectorized with calculated NaN values, this initially led to exceptions being thrown from ASGI, now that we catch them it blows up less spectacularly but still does.
raam86
commented
2 years ago
Naive addition of MPS support for the transformers service.
At the moment the service crashes python when load is high, still looking into what qualifies as high but it seems like pytorch is crashing python due to some MPS issue.
The issue manifests as some embeddings are vectorized with calculated NaN values, this initially led to exceptions being thrown from ASGI, now that we catch them it blows up less spectacularly but still does.