search

openwebf / webf

Build flutter apps with HTML/CSS and JavaScript.
https://openwebf.com/
Apache License 2.0
1.66k stars 121 forks source link

npm run test出错。libwebf.so: undefined symbol: initTestFramework #642

Closed linsmod closed 3 months ago

linsmod commented 3 months ago

Affected version

0.16.1

Flutter versions

3.19.6

No same issues found.

Steps to Reproduce

npm run test

Code example

npm run test

Expected results

pass the build and run with no errors.

Actual results

Building Linux application...                                   
flutter: The Dart VM service is listening on http://127.0.0.1:41679/MFowDgw6j-k=/
Mock HTTP server listening on port 4567
Separate HTTP server listening on port 4568
flutter: Local HTTP server started at: http://127.0.0.1:18338
flutter: Invalid argument(s): Failed to lookup symbol 'initTestFramework': /home/wulin/webf/integration_tests/build/linux/x64/debug/bundle/lib/libwebf.so: undefined symbol: initTestFramework
#0      DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:33:70)
#1      _initTestFramework (package:webf_integration_tests/bridge/to_native.dart:25:32)
#2      _initTestFramework (package:webf_integration_tests/bridge/to_native.dart)
#3      initTestFramework (package:webf_integration_tests/bridge/to_native.dart:28:10)
#4      _WebFTesterState.onControllerCreated (package:webf_integration_tests/webf_tester.dart:87:19)
#5      WebFRootRenderObjectWidget.createRenderObject.<anonymous closure> (package:webf/src/widget/webf.dart:402:28)
#6      _rootRunUnary (dart:async/zone.dart:1407:47)
#7      _CustomZone.runUnary (dart:async/zone.dart:1308:19)
#8      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:838:45)
#9      Future._propagateToListeners (dart:async/future_impl.dart:867:13)
#10     Future._propagateToListeners (dart:async/future_impl.dart:763:9)
#11     Future._completeWithValue (dart:async/future_impl.dart:643:5)
#12     Future._asyncCompleteWithValue.<anonymous closure> (dart:async/future_impl.dart:713:7)
#13     _rootRun (dart:async/zone.dart:1399:13)
#14     _CustomZone.run (dart:async/zone.dart:1301:19)
#15     _CustomZone.runGuarded (dart:async/zone.dart:1209:7)
#16     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1249:23)
#17     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#18     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
ReferenceError: 'afterEach' is not defined
    at ./runtime/global.ts (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:287:1)
    at call (native)
    at __webpack_require__ (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:142229:42)
    at <anonymous> (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:142305:11)
    at <eval> (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:142950:10)
flutter: Null check operator used on a null value
andycall commented 3 months ago

only support macOS platform.

linsmod commented 3 months ago

还是好奇为什么会出现这个问题，Dart不是跨平台的嘛，那么理论上不会出现这样的问题才对。

核实了代码之后，发现linux这边测试的使用的so文件名指定的是libwebf， 和windows，macos指定的libwebf_test不一致。

我修正为libwebf_test之后又出现了新的问题，不理解。

undefined symbol: Dart_NewPersistentHandle_DL

flutter: Local HTTP server started at: http://127.0.0.1:45129
flutter: Invalid argument(s): Failed to load dynamic library '$ORIGIN/libwebf_test.so': 
/home/wulin/webf/integration_tests/build/linux/x64/debug/bundle/lib/libwebf_test.so: 
undefined symbol: Dart_NewPersistentHandle_DL
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:11:43)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:22:12)
#2      WebFDynamicLibrary.testRef (package:webf/src/bridge/dynamic_library.dart:63:56)
#3      _initTestFramework (package:webf_integration_tests/bridge/to_native.dart:25:24)
#4      _initTestFramework (package:webf_integration_tests/bridge/to_native.dart)
#5      initTestFramework (package:webf_integration_tests/bridge/to_native.dart:28:10)
#6      _WebFTesterState.onControllerCreated (package:webf_integration_tests/webf_tester.dart:87:19)
#7      WebFRootRenderObjectWidget.createRenderObject.<anonymous closure> (package:webf/src/widget/webf.dart:402:28)
#8      _rootRunUnary (dart:async/zone.dart:1407:47)
#9      _CustomZone.runUnary (dart:async/zone.dart:1308:19)
#10     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:838:45)
#11     Future._propagateToListeners (dart:async/future_impl.dart:867:13)
#12     Future._propagateToListeners (dart:async/future_impl.dart:763:9)
#13     Future._completeWithValue (dart:async/future_impl.dart:643:5)
#14     Future._asyncCompleteWithValue.<anonymous closure> (dart:async/future_impl.dart:713:7)
#15     _rootRun (dart:async/zone.dart:1399:13)
#16     _CustomZone.run (dart:async/zone.dart:1301:19)
#17     _CustomZone.runGuarded (dart:async/zone.dart:1209:7)
#18     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1249:23)
#19     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#20     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
ReferenceError: 'afterEach' is not defined
    at ./runtime/global.ts (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:287:1)
    at call (native)
    at __webpack_require__ (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:142312:42)
    at <anonymous> (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:142388:11)
    at <eval> (http://localhost:4567/public/core.build.js?search=1234#hash=hashValue:143034:10)
flutter: Null check operator used on a null value
andycall commented 3 months ago

integration 依赖一个特殊构建的 .so 产物，我比较懒，没有写针对 android 的构建脚本，而且android 设备尺寸不统一，也不方便去启动各种本地服务。

linsmod commented 3 months ago

我构建的linux版本，不是安卓，也有影响吗。

linux这边测试的使用的so文件名指定的是libwebf， 和windows，macos指定的libwebf_test不一致。

Linux也改成testLibName，记录到这里备忘。

  static String get _nativeDynamicLibraryTestName {
    if (Platform.isMacOS) {
      return 'lib$testLibName.dylib';
    } else if (Platform.isWindows) {
      return '$testLibName.dll';
    } else if (Platform.isLinux) {
      return 'lib$testLibName.so';
    } else {
      throw UnimplementedError('Not supported platform.');
    }
  }
andycall commented 3 months ago

Linux 应该也可以，不过没有很多测试

andycall commented 3 months ago

不过思路都是一样，先是构建 webf_test 来产出 libwebf_test.so 然后启动 integration test 工程

linsmod commented 3 months ago

改了一点点，之前的错误没有了： 1、添加${BRIDGE_SOURCE} 2、定义DART_SHARED_LIB

add_library(webf_test SHARED ${WEBF_TEST_SOURCE} ${BRIDGE_SOURCE})
target_link_libraries(webf_test PRIVATE ${BRIDGE_LINK_LIBS} webf)
target_compile_definitions(webf_test PUBLIC -DDART_SHARED_LIB)

但是新的错误。

flutter: Local HTTP server started at: http://127.0.0.1:14467
Error: signal 11:
libwebf_test.so(_Z7handleri+0x35)[0x737e6b1193b5]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x737f1ea42520]
libquickjs.so(JS_ExecutePendingJob+0x1c)[0x737e741580ac]
libwebf_test.so(_ZN4webf16ExecutingContext23DrainPendingPromiseJobsEv+0x9b)[0x737e6b13c45b]
libwebf_test.so(_ZN4webf16ExecutingContext15DrainMicrotasksEv+0x72)[0x737e6b13c5a2]
libwebf_test.so(_ZN4webf16ExecutingContext16EvaluateByteCodeEPhm+0x180)[0x737e6b13f9e0]
libwebf_test.so(_Z21initWebFTestFrameworkPN4webf16ExecutingContextE+0x19)[0x737e6b11e6b9]
libwebf_test.so(_ZN4webf15WebFTestContextC1EPNS_16ExecutingContextE+0x1db)[0x737e6b11fc3b]
libwebf_test.so(+0x11fcd5)[0x737e6b11fcd5]
libwebf_test.so(_ZNSt13__future_base13_State_baseV29_M_do_setEPSt8functionIFSt10unique_ptrINS_12_Result_baseENS3_8_DeleterEEvEEPb+0x32)[0x737e6b119012]
wulin@R7000:~/webf$

gdb看，似乎是JSRuntime没有初始化。

Thread 41 "io.flutter.ui" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff522ff640 (LWP 2438281)]
JS_ExecutePendingJob (rt=0x0, pctx=0x7fff522fe5a8) at /home/wulin/webf/bridge/third_party/quickjs/src/core/runtime.c:1685
1685      if (list_empty(&rt->job_list)) {
andycall commented 3 months ago

不能这么改，把 BRIDGE SOURCE 加到 webf_test 这个 target 内，JSRuntime 就跑到这里初始化了，应用那边就访问不到了

andycall commented 3 months ago

libwebf_test.so 在 linux 本地平台应该是有的

linsmod commented 3 months ago

libwebf_test.so 在 linux 本地平台应该是有的

有是有的，动态载入的时候就各种undefined symbol。

不能这么改，把 BRIDGE SOURCE 加到 webf_test 这个 target 内，JSRuntime 就跑到这里初始化了，应用那边就访问不到了

确实，我看了下出错那里是渲染线程向js线程提交过来的task，这时候不知道是时机问题还是隔离的原理，JSRuntime没有初始化。

andycall commented 3 months ago

奇怪了，载入顺序竟然和 macOS 不一样。Dart_NewPersistentHandle_DL 这个的使用前提依赖 Dart FFI 将函数指针集合发过来完成初始化。

https://github.com/openwebf/webf/blob/main/bridge/webf_bridge.cc#L300

估计是这里还没执行

linsmod commented 3 months ago

undefined symbol: Dart_NewPersistentHandle_DL是因为没有导入dartAPI，定义 DART_SHARED_LIB就没有这个错误了。然后是undefined symbol: xxxx, 就是找不到codeGen生成的out里面的c或h定义的那个static const WrapperTypeInfo wrapper_typeinfo;，所以我才添加${BRIDGE_SOURCE}

linsmod commented 3 months ago

这样之后，编译出来就是似乎JSRuntime没有初始化，我分析了一下调用堆栈，发现好像是初始化JSRuntime的时机在渲染线程之后，或者是被隔离了

linsmod commented 3 months ago 
这是初始化JSRuntime的堆栈。
JS_NewRuntime
--InitializeJSRuntime is before dispatcher_->PostToDart(true, HandleNewPageResult
----InitializeNewPageInJSThread
------AddNewPage
--------allocateNewPage
--------fn:HandleNewPageResult by dispatcher_->PostToDart
----------fn:InitializeNewPageInJSThread by dispatcher_->PostToJs
-----------AddNewPage

----InitializeNewPageSync
------AddNewPageSync
--------allocateNewPageSync
----------TEST_init
----------TEST_allocateNewPage
----------_allocateNewPageSync
------------allocateNewPage

这里是使用JSRuntime的堆栈。
Here use rt but it's nullptr
-JS_ExecutePendingJob
--DrainPendingPromiseJobs
----DrainMicrotasks
------EvaluateByteCode
--------initWebFTestFramework
----------fn=WebFTestContext::WebFTestContext by dispatcher()->PostToJsSync
------------initTestFramework
--------------onControllerCreated
----------------createRenderObject

这是线程初始化。
-Looper::Run()
--Looper::Start()
---AllocateNewJSThread
-----AddNewPage
andycall commented 3 months ago

把 webf 的动态库加载也改成 libwebf_test.so 试试看

https://github.com/openwebf/webf/blob/main/webf/lib/src/bridge/dynamic_library.dart#L28

linsmod commented 3 months ago

这边已经是这样了，使用 libwebf_test.so是最先改好的。

linsmod commented 3 months ago

JSRuntime是全局的是吗，不会因为不同线程去初始化，就产生多个独立于线程的实例对吗

andycall commented 3 months ago

https://github.com/openwebf/webf/blob/main/bridge/core/dart_isolate_context.cc#L63

是线程独立的

linsmod commented 3 months ago

现在不使用${BRIDGE_SOURCE}:

add_library(webf_test SHARED ${WEBF_TEST_SOURCE})

正如前面说的，会出现找不到另一个符号的错误，看看有没有思路：

flutter: Invalid argument(s): Failed to load dynamic library '$ORIGIN/libwebf_test.so': 
/home/wulin/webf/integration_tests/build/linux/x64/debug/bundle/lib/libwebf_test.so: 

undefined symbol: _ZN4webf7QJSBlob18wrapper_type_info_E
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:11:43)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:22:12)
#2      WebFDynamicLibrary.testRef (package:webf/src/bridge/dynamic_library.dart:63:56)
#3      _initTestFramework (package:webf_integration_tests/bridge/to_native.dart:25:24)
#4      _initTestFramework (package:webf_integration_tests/bridge/to_native.dart)
#5      initTestFramework (package:webf_integration_tests/bridge/to_native.dart:28:10)
#6      _WebFTesterState.onControllerCreated (package:webf_integration_tests/webf_tester.dart:87:19)
#7      WebFRootRenderObjectWidget.createRenderObject.<anonymous closure> (package:webf/src/widget/webf.dart:402:28)
#8      _rootRunUnary (dart:async/zone.dart:1407:47)
#9      _CustomZone.runUnary (dart:async/zone.dart:1308:19)
#10     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:838:45)
#11     Future._propagateToListeners (dart:async/future_impl.dart:867:13)
#12     Future._propagateToListeners (dart:async/future_impl.dart:763:9)
#13     Future._completeWithValue (dart:async/future_impl.dart:643:5)
#14     Future._asyncCompleteWithValue.<anonymous closure> (dart:async/future_impl.dart:713:7)
#15     _rootRun (dart:async/zone.dart:1399:13)
#16     _CustomZone.run (dart:async/zone.dart:1301:19)
#17     _CustomZone.runGuarded (dart:async/zone.dart:1209:7)
#18     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1249:23)
#19     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#20     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)

而且qjs_blob.cc也是编译到webf内的，没道理找不到符号的。

namespace webf {
const WrapperTypeInfo QJSBlob::wrapper_type_info_ {JS_CLASS_BLOB, "Blob", nullptr, QJSBlob::ConstructorCallback};
const WrapperTypeInfo& Blob::wrapper_type_info_ = QJSBlob::wrapper_type_info_;
linsmod commented 3 months ago

没有搞定，但是小结一下。

nm 工具的输出显示 libwebf_test.so 依赖于 _ZN4webf7QJSBlob18wrapper_type_info_E 这个符号，并且该符号在 libwebf_test.so 中被标记为 U，即 Undefined（未定义）。这意味着 libwebf_test.so 需要在运行时从其他库中找到这个符号的定义。

幸运的是，从 ldd 的输出中我们可以看到 libwebf_test.so 依赖于 libwebf.so，并且 nm 的输出也表明 _ZN4webf7QJSBlob18wrapper_type_info_E 确实存在于 libwebf.so 中。因此，当 libwebf_test.so 被动态加载时，动态链接器（如 ld.so）将能够解析这个未定义的符号，因为它存在于 libwebf.so 中。

总结来说，是的，libwebf_test.so 在动态加载时能够找到 _ZN4webf7QJSBlob18wrapper_type_info_E，因为这个符号的定义存在于它依赖的 libwebf.so 库中。

故而怀疑是FFI的问题。