facebookresearch / fastText

Library for fast text representation and classification.
https://fasttext.cc/
MIT License
25.76k stars 4.71k forks source link

Unable to build wasm #1317

Open Cameron-Evans opened 1 year ago

Cameron-Evans commented 1 year ago

Hi, I'm unable to get the wasm version to build correctly. It looks like an issue with function definitions. I've removing "EXTRA_" from the build flag and that doesn't change the main issue of the undefined symbols. Any help would be appreciated.

# em++ --version

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.30 (cfe2bdfe2692457cb5f5770672f6e5ccb3ffc2f2)
Copyright (C) 2014 the Emscripten authors (see AUTHORS.txt)
This is free and open source software under the MIT license.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

# git clone https://github.com/facebookresearch/fastText.git
# cd fastText
# make wasm
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/args.cc -o args.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/autotune.cc -o autotune.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/ src/matrix.cc -o matrix.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/dictionary.cc -o dictionary.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/ src/loss.cc -o loss.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/productquantizer.cc -o productquantizer.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/ src/densematrix.cc -o densematrix.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/ src/quantmatrix.cc -o quantmatrix.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/vector.cc -o vector.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/model.cc -o model.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/utils.cc -o utils.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/meter.cc -o meter.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  src/fasttext.cc -o fasttext.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
em++ --bind --std=c++11 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['addOnPostRun', 'FS']" -s "DISABLE_EXCEPTION_CATCHING=0" -s "EXCEPTION_DEBUG=1" -s "FORCE_FILESYSTEM=1" -s "MODULARIZE=1" -s "EXPORT_ES6=1" -s 'EXPORT_NAME="FastTextModule"' -Isrc/  webassembly/fasttext_wasm.cc -o main.bc
em++: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
em++: warning: object file output extension (.bc) used for non-object output.  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::predictLine(std::__2::basic_istream<char, std::__2::char_traits<char>>&, std::__2::vector<std::__2::pair<float, std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char>>>, std::__2::allocator<std::__2::pair<float, std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char>>>>>&, int, float) const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getDimension() const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Vector::Vector(long long)
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getWordVector(fasttext::Vector&, std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char>> const&) const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getDimension() const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Vector::Vector(long long)
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getSentenceVector(std::__2::basic_istream<char, std::__2::char_traits<char>>&, fasttext::Vector&)
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getDictionary() const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Dictionary::getSubwords(std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char>> const&, std::__2::vector<int, std::__2::allocator<int>>&, std::__2::vector<std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char>>, std::__2::allocator<std::__2::basic_string<char, std::__2::char_traits<char>, std::__2::allocator<char>>>>&) const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getDimension() const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Vector::Vector(long long)
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Vector::zero()
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::addInputVector(fasttext::Vector&, int) const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::train(fasttext::Args const&, std::__2::function<void (float, float, double, double, long long)> const&)
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getInputMatrix() const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getOutputMatrix() const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::FastText::getDictionary() const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Dictionary::getCounts(fasttext::entry_type) const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Dictionary::getWord(int) const
wasm-ld: error: /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o: undefined symbol: fasttext::Dictionary::getLabel(int) const
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
em++: error: '/root/emsdk/upstream/bin/wasm-ld -o main.wasm /tmp/emscripten_temp_68_18aqk/fasttext_wasm_0.o -L/root/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten --whole-archive -lembind-rtti --no-whole-archive -lGL -lal -lhtml5 -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lcompiler_rt -lc++ -lc++abi-debug -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined-file=/tmp/tmp4d9m2m3_.undefined --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_get_current --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export=getTempRet0 --export=setTempRet0 --export=__get_temp_ret --export=__set_temp_ret --export=malloc --export=free --export=__cxa_is_pointer_type --export=__cxa_can_catch --export=setThrew --export=__cxa_free_exception --export-table -z stack-size=65536 --initial-memory=16777216 --no-entry --max-memory=2147483648 --stack-first' failed (returned 1)
make: *** [Makefile:81: main.bc] Error 1
tianfeng98 commented 1 year ago

Help! I have the exact same problem.

System: MacOS 13.2.1

jincdream commented 1 year ago

+1

dragonman225 commented 1 year ago

I have the same issue on macOS 13.4.1 (M1)

Sejmou commented 11 months ago

So, has anyone figured out what the issue is? It also fails on my M1 Max MacBook Pro (macOS 13.4.1). Is this an ARM vs. x86 issue?

yunsii commented 10 months ago

It also build fails on: RELEASE: Ubuntu 18.04.6 LTS KERNEL: Linux 5.15.90.1-microsoft-standard-WSL2

yunsii commented 10 months ago

Use Xmake save my life, after build wasm, I create a lib fasttext.wasm.js to use fastText in Node.js and browser. Xmake config: https://github.com/yunsii/fasttext.wasm.js/blob/master/xmake.lua