search

tonybaloney / Pyjion

Pyjion - A JIT for Python based upon CoreCLR
https://www.trypyjion.com
MIT License
1.42k stars 59 forks source link

Export of Python function to a .NET assembly file #503

Closed vadimkantorov closed 1 year ago

vadimkantorov commented 1 year ago

Hy! My scenario is to use a rather simple Python function from .NET.

As far as I understood, this is what Pyjion does anyway: produces IL and saves a .NET assembly. Is it possible to obtain this assembly / clean way of calling it from C# / .NET? Does Pyjion provide an explicit API of exporting a Python function to a .NET assembly?

Thanks!

tonybaloney commented 1 year ago

Hey

Unfortunately no, this isn't possible using Pyjion. But you can do it using this other project

https://pythonnet.github.io/pythonnet/dotnet.html

vadimkantorov commented 1 year ago

It seems that PythonNet is actually embedding CPython, right?

I was wondering whether the CIL / Pyjion-generated assemblies can be used from .NET as well (as it seems a cleaner solution if the original Python code is sufficiently simple)

Is it not possible because the interfacing with the generated assemblies is hard / needs extra runtime?

Because it seemed that if you generate assemblies anyway, some marshalling and function calling these assemblies from C#/.NET should be possible, no?

vadimkantorov commented 1 year ago

Should we just be able to dump m_il byte code to an assembly?

or maybe first create MethodInfo and then create an assembly just with one method?

a question: will pyjion jit methods called from the currently-jitted method? (including the python standard library)

An example of creating a minimal assembly from a MethodInfo seems here in the JitPacker/Unpacker: project https://github.com/LJP-TW/JITHook and slides deck: https://jsac.jpcert.or.jp/archive/2023/pdf/JSAC2023_1_4_chang_en.pdf

tonybaloney commented 1 year ago

I spent some time looking at the reverse. There are lots of issues to overcome.

Firstly, and most importantly, Pyjion only uses RyuJIT. We never initialise the managed .NET runtime. That in itself is a huge undertaking. Secondly the object models are not compatible. Pyjion uses low level types for integers, floats, bool. Beyond that it doesn't implement any managed types/classes in CIL because they aren't compatible with Pythons objects, nor would there be any advantage to doing so. Third, as you pointed out you need export more information into an assembly that can be imported and run from .NET. I started investigating it but decided it was too much work.

vadimkantorov commented 1 year ago

I see, understand it better now. Thank you!