Closed gusye1234 closed 1 year ago
Actually, yes, that has been a bit of a challenge for @brianjjones who has been working on a TensorFlow implementation of wasi-nn in Wasmtime. There have been some hoops to jump through, e.g., here, and perhaps some examination of other parts of that tf.rs
file should show how he resolved the problem.
I would like to point out one additional difficulty with wasi-nn and TF: wasi-nn expects to be able to receive the model, weights, etc. as raw byte buffers but, AFAICT, TF expects to have access to the file system for reading a set of files. This is a bit tricky in a WASI environment, since it relies on the user having made that directory a preopened directory, which may not be immediately apparent. Furthermore, it is then odd that wasi-nn models using OpenVINO, e.g., can be initialized with byte buffers but in the TF case these byte buffers are file paths — inconsistent usage of the API. I see two ways around this: a) TF adds methods for initializing models from byte buffers, or b) we add a new wasi-nn mechanism to pre-initialize models prior to the Wasm module's instantiation; pre-initialized models would then be referred to by name. This second method might be useful elsewhere so I have been leaning that direction — any thoughts?
Thanks for your guide, will look through this PR
TF expects to have access to the file system for reading a set of files
I decided to only load .pb
file😂 so that it can still pass the raw bytes to wasi-nn and create a graph. Not familiar with tensorflow-rs, I guess that could also be a workaround for wasmtime
. But yeah, that's another difficulty.
I'm currently mantaining a wasi-nn implementation in WAMR (written in C) where we support TFLite;
I am going to close this. At the time this was written, users can use TensorFlow as their wasi-nn backend by:
Noting this here as well so people are aware of security implications of using tensorflow with wasi-nn https://github.com/bytecodealliance/wasmtime/pull/3977#issuecomment-1533550546
Hi, I'm trying to support TensorFlow backend for Wasi-NN, but encountered several problems. To specify, the C API's version is
2.6.0
https://github.com/tensorflow/tensorflow/blob/919f693420e35d00c8d0a42100837ae3718f7927/tensorflow/c/c_api.h#L688-L689
https://github.com/tensorflow/tensorflow/blob/919f693420e35d00c8d0a42100837ae3718f7927/tensorflow/c/c_api.h#L1351-L1357
I noticed that you guys may have plans to support Tensorflow. Have you encountered those issues? And if the above two points about Tensorflow are correct, do you think Wasi-NN should adapt its interfaces?