hmenyus / node-calls-python

Call Python from NodeJS directly in-process without spawning processes
MIT License
252 stars 26 forks source link

Feature Request: Support TypedArrays #85

Closed fusir closed 5 months ago

fusir commented 5 months ago

I like ML opportunities this presents. Though in ML we are often working with arrays with different float sizes. This also helps with serializing or mapping data to disk.

I tried passing it a Float32Array as a test. It converted it to an object with the indexes as keys.

What would be very cool is if it converted it to a nparray(dtype=np.float32);

Even cooler would be torch.tensor(dtype=torch.float32);

Even cooler than that would be to be able to write custom type converters. It is possible in napi to get the direct pointer to the data of a TypedArray. I'm not a python expert, hence my interest in javascript libraries that can give us access to python modules, but I think there are ways to represent a c-pointer in python. That is something that could be handed to an extensions.py where extensions.py is user defined and could handle the type conversion themselves.

But maybe that is asking too much. I think the conversion to a dtyped nparray though would be pretty straightforward and meet most people's needs. In fact a TypedArray is probably a good target for python to javascript.

hmenyus commented 5 months ago

Proper conversion of typedarrays to bytes are now available in 1.10.0. Please see some examples here: https://github.com/hmenyus/node-calls-python/blob/b3b002b71972fe3c766433402b2fadcf97267e4f/test/test.js#L242