search

pytorch / multipy

torch::deploy (multipy for non-torch uses) is a system that lets you get around the GIL problem by running multiple Python interpreters in a single C++ process.
Other
174 stars 35 forks source link

Question: how much do we support non-PyTorch code like NumPy #301

Closed KexinFeng closed 1 year ago

KexinFeng commented 1 year ago

This is a follow up question to the original question https://github.com/pytorch/multipy/issues/296. It is basically the same as the comments in https://github.com/pytorch/multipy/issues/296. But it is closed, so I don't know the comments there will be noticed. Sorry about this repetition.

In the original question, it is mentioned that the an interface for NumPy is needed to be registered into the interpreter. But it seems that, in the test examples, a python script can still have numpy dependency and is still working. For example, here https://github.com/pytorch/multipy/blob/main/README.md#packaging-a-model-for-multipyruntime the numpy packages installed in the system will be searched.

Does this mean NumPy is currently still somewhat supported by MultiPy? It is like, as long as I have numpy package installed, the python interpreters in MultiPy will load them during runtime and still be able to do multi-threading inference?

Here is another example, it can be imported with following code I.global("numpy", "random"), which is from https://github.com/pytorch/multipy/blob/19617b95252c97547e17d598c0b2cb03d0f4e936/multipy/runtime/test_deploy.cpp#L494-L504 Given this, I'm wondering: what is the effect/purpose of registering the NumPy interface and

get numpy to convert to and from IValue

as was mentioned in question https://github.com/pytorch/multipy/issues/296. What does this step do?

PaliC commented 1 year ago

Yup this is exactly it. We just use the type to interact with the interpreters. For numpy we haven't done thorough testing, so we can't provide any guarantees. Though you're right in that things should generally just work (IValue does cover a lot haha just not everything).

For the plugins/convertors (the interface I think you're referring to), currently we use IValue as an intermediary to convert a pyobject to something usable in C++. For example on line 501 you go from pyobject->IValue->int. However, eventually we'd like to create a custom convertor get more coverage.

Sorry to be more clear if IValue works for your use cases feel free to use it. However, if there are objects which you can't get out of IValue, you'd want to write your own convertor.

Let's continue this conversation in #296 . I'll copy and paste this reply there.