joshhansen
commented
1 year ago I seem to be running into this with tch 0.13 running inference in a Tokio async environment, where the models can't be guaranteed to allocate on the same thread as the inference runs on.
At this point it would be hard to move my code away from async.
Apparently this is a libtorch issue in general:
Hello, thank you a lot for this lib.
I've been using it to run inference on the YoloV3 example. However, after inference the thread local variables seem to not be cleaned up.
To reproduce the issue, just run Yolo and check that the thread now uses around ~200MB of memory, even if it is just sleeping.
The issue seem to be the same as https://github.com/pytorch/pytorch/issues/24237#issuecomment-558882706 . I wonder if there is something we can do about it. Anyway, a warning on the README would be nice to warn people about this.