Closed ipetr0v closed 4 years ago
ipetr0v
commented
4 years ago I tried to use assemblyscript to compile TypeScript to WebAssenbly, but it didn't work for TensorFlow Lite, since assemblyscript can only compile a strong subset of TypeScript, and TensorFlow Lite uses too many language features.
ipetr0v
commented
4 years ago Currently I was able to compile examples from TensorFlow Lite to WebAssembly with emscripten toolchain using the following flags:
"-s WASM=1",
"-s AUTO_ARCHIVE_INDEXES=0",
"-s ALLOW_MEMORY_GROWTH=1",
"-s DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=[]",
"-s DISABLE_EXCEPTION_CATCHING=1",
"-s FILESYSTEM=0",
"-s EXIT_RUNTIME=0",
"-s EXPORTED_FUNCTIONS='[\"_main\"]'",
"-s MODULARIZE=1",
"-s MALLOC=emmalloc",
"-s LINKABLE=1",
tiziano88
commented
4 years ago @ipetr0v thanks, that's great progress! In the Wasm generated file, did you see what imported / exported symbols are defined? You can use wasm-objdump for that.
ipetr0v
commented
4 years ago It exports multiple TF functions like:
...
(export "_ZN14EigenForTFLite19ThreadPoolInterface6CancelEv" (func 1510))
...$ cat bazel-bin/tensorflow/lite/examples/label_image/label_image.wat|grep export|grep TF|wc -l
171how about imports? those are the problematic ones, since they need to be provided by the runtime
ipetr0v
commented
4 years ago Currently it has a lot of imports from standard libraries:
(import "env" "gettimeofday" (func (;0;) (type 1)))
(import "env" "roundf" (func (;1;) (type 36)))
(import "env" "round" (func (;2;) (type 29)))
(import "env" "pthread_cond_init" (func (;3;) (type 1)))
(import "env" "pthread_create" (func (;4;) (type 6)))
(import "env" "pthread_equal" (func (;5;) (type 1)))
(import "env" "sysconf" (func (;6;) (type 2)))
(import "env" "__powidf2" (func (;7;) (type 73)))
(import "env" "pthread_join" (func (;8;) (type 1)))
(import "env" "pthread_cond_destroy" (func (;9;) (type 2)))
(import "env" "dlopen" (func (;10;) (type 1)))
(import "env" "dlsym" (func (;11;) (type 1)))
(import "wasi_snapshot_preview1" "args_sizes_get" (func (;12;) (type 1)))
...Also a lot of targets in TFLite use -lpthread, that also might be a problem for us.
tiziano88
commented
4 years ago It seems these functions fall in one of these categories:
getting external inputs (time, env variables, command line args): as long as they are configuration options (and not, for instance, trying to retrieve user data to process), it should be possible to define these symbols in a C++ file and link it together with the rest of the code, so that they are not required as imports any more. I am not sure whether emscipten actually works this way though.
@ipetr0v could you try defining a gettimeofday function with the correct type and returning 0 or another const value and link it against the rest of the code, and see if the import for that symbol disappears from the resulting Wasm file?
threads: when Wasm implements threads natively (https://github.com/WebAssembly/proposals/issues/14), these imports should be unnecessary (I hope), so we can probably just wait and see what happens there
math: these also look like they could be inlined by taking existing implementations from musl or newlib or some other canonical libc implementation, and hopefully by linking them together, the imports should disappear
dlopen / dlsym: not sure why the code relies on dynamic loading, and not sure how these are even implemented in Wasm when running in the browser. @ipetr0v do you know more about them? Are they actually used at runtime?
ipetr0v
commented
4 years ago gettimeofday was used by the example itself:
https://github.com/tensorflow/tensorflow/blob/8e83ed87294beefe8e2aaefdffdc4384117dec05/tensorflow/lite/examples/label_image/label_image.cc#L256-L267
I compiled an empty main.cc with TFLite dependencies and got these imports:
(import "env" "roundf" (func (;0;) (type 36)))
(import "env" "round" (func (;1;) (type 29)))
(import "env" "pthread_cond_init" (func (;2;) (type 1)))
(import "env" "pthread_create" (func (;3;) (type 6)))
(import "env" "pthread_equal" (func (;4;) (type 1)))
(import "env" "sysconf" (func (;5;) (type 3)))
(import "env" "__powidf2" (func (;6;) (type 72)))
(import "env" "pthread_join" (func (;7;) (type 1)))
(import "env" "pthread_cond_destroy" (func (;8;) (type 3)))
(import "env" "dlopen" (func (;9;) (type 1)))
(import "env" "dlsym" (func (;10;) (type 1)))
(import "wasi_snapshot_preview1" "args_sizes_get" (func (;11;) (type 1)))
(import "wasi_snapshot_preview1" "args_get" (func (;12;) (type 1)))
(import "wasi_snapshot_preview1" "proc_exit" (func (;13;) (type 4)))
(import "env" "__syscall197" (func (;14;) (type 1)))
(import "env" "__syscall195" (func (;15;) (type 1)))
(import "env" "__syscall5" (func (;16;) (type 1)))
(import "wasi_snapshot_preview1" "fd_close" (func (;17;) (type 3)))
(import "env" "__syscall221" (func (;18;) (type 1)))
(import "env" "__syscall54" (func (;19;) (type 1)))
(import "wasi_snapshot_preview1" "fd_seek" (func (;20;) (type 134)))
(import "wasi_snapshot_preview1" "fd_write" (func (;21;) (type 6)))
(import "wasi_snapshot_preview1" "fd_read" (func (;22;) (type 6)))
(import "env" "__syscall194" (func (;23;) (type 1)))
(import "env" "pthread_setcancelstate" (func (;24;) (type 1)))
(import "env" "__syscall10" (func (;25;) (type 1)))
(import "env" "__syscall192" (func (;26;) (type 1)))
(import "wasi_snapshot_preview1" "fd_fdstat_get" (func (;27;) (type 1)))
(import "env" "clock_gettime" (func (;28;) (type 1)))
(import "env" "pthread_cond_timedwait" (func (;29;) (type 2)))
(import "env" "pthread_mutexattr_init" (func (;30;) (type 3)))
(import "env" "pthread_mutexattr_settype" (func (;31;) (type 1)))
(import "env" "pthread_mutexattr_destroy" (func (;32;) (type 3)))
(import "env" "pthread_detach" (func (;33;) (type 3)))
(import "env" "nanosleep" (func (;34;) (type 1)))
(import "env" "sched_yield" (func (;35;) (type 7)))
(import "env" "invoke_vii" (func (;36;) (type 0)))
(import "env" "__cxa_find_matching_catch_2" (func (;37;) (type 7)))
(import "env" "getTempRet0" (func (;38;) (type 7)))
(import "env" "__resumeException" (func (;39;) (type 4)))
... and a lot more ...It looks like dlopen are there because TensorFlow allows to compile some parts a .so files and load them dynamically:
https://github.com/tensorflow/tensorflow/blob/8e83ed87294beefe8e2aaefdffdc4384117dec05/tensorflow/core/framework/load_library.cc#L36-L47
ipetr0v
commented
4 years ago I added a -s LINKABLE=0 parameter, that tells emscripten to delete the dead code, and these are the only imports left:
cat bazel-bin/tensorflow/lite/examples/minimal/minimal.wat|grep import
(import "env" "dlsym" (func (;0;) (type 1)))
(import "env" "roundf" (func (;1;) (type 23)))
(import "env" "pthread_equal" (func (;2;) (type 1)))
(import "env" "sysconf" (func (;3;) (type 4)))
(import "env" "round" (func (;4;) (type 29)))
(import "wasi_snapshot_preview1" "proc_exit" (func (;5;) (type 3)))
(import "env" "__powidf2" (func (;6;) (type 37)))
(import "env" "pthread_create" (func (;7;) (type 10)))
(import "env" "pthread_cond_destroy" (func (;8;) (type 4)))
(import "env" "pthread_join" (func (;9;) (type 1)))
(import "env" "pthread_setcancelstate" (func (;10;) (type 1)))
(import "wasi_snapshot_preview1" "fd_close" (func (;11;) (type 4)))
(import "env" "dlopen" (func (;12;) (type 1)))
(import "env" "emscripten_notify_memory_growth" (func (;13;) (type 3)))
(import "wasi_snapshot_preview1" "environ_get" (func (;14;) (type 1)))
(import "wasi_snapshot_preview1" "environ_sizes_get" (func (;15;) (type 1)))
(import "env" "nanosleep" (func (;16;) (type 1)))
(import "env" "clock_gettime" (func (;17;) (type 1)))
(import "env" "__syscall192" (func (;18;) (type 1)))
(import "env" "pthread_cond_init" (func (;19;) (type 1)))
(import "env" "__syscall194" (func (;20;) (type 1)))
(import "wasi_snapshot_preview1" "fd_write" (func (;21;) (type 10)))
(import "wasi_snapshot_preview1" "fd_seek" (func (;22;) (type 89)))
(import "env" "__syscall5" (func (;23;) (type 1)))
(import "wasi_snapshot_preview1" "args_get" (func (;24;) (type 1)))
(import "wasi_snapshot_preview1" "args_sizes_get" (func (;25;) (type 1)))It also started to show the following warning:
warning: undefined symbol: __powidf2could you still try to define one of those symbols (e.g. args_get) somewhere in your example code as extern (even alongside main should work), and see if it disappears from that list?
ipetr0v
commented
4 years ago I added:
extern void *dlopen(const char *filename, int flags);but dlopen is still present:
(import "env" "dlopen" (func (;12;) (type 1)))Isn't that just a declaration though? it would still just require the symbol to be provided externally. I think you have to provide an actual definition (even empty) in order to bind the symbol. I may be wrong though.
ipetr0v
commented
4 years ago This also doesn't change the list of imports:
extern void *dlopen(const char *filename, int flags) {
return 0x0;
}Does it need extern "C" by any chance?
ipetr0v
commented
4 years ago That worked:
extern "C" void *dlopen(const char *filename, int flags) {
return 0x0;
}Now dlopen is not in the list (and even dlsym is not in the list too).
tiziano88
commented
4 years ago If it's not too much work, it would be useful to define all the missing symbols and make them trap (not sure how to write a C++ statement that results in a Wasm trap, but there must be a way), and see if and when these symbols are actually exercised at runtime.
We need to create an example Oak application that uses TensorFlow Lite. In order to do this we need to compile TensorFlow Lite in WebAssembly.
Currently we have 3 implementation options:
Model training in TensorFlow Lite is not yet supported. Feature request is tracked in https://github.com/tensorflow/tensorflow/issues/27646.