GregoryConrad / mimir

⚡ Supercharged Flutter/Dart Database
https://pub.dev/packages/mimir
MIT License
120 stars 8 forks source link

Running tests fails - possible config issue #99

Closed CicadaCinema closed 1 year ago

CicadaCinema commented 1 year ago

Hi, I'm trying to use mimir to better understand the guide you wrote for flutter_rust_bridge.

I am facing an issue which I can replicate with mimir:

atom@Dell-Desktop:~$ git clone https://github.com/GregoryConrad/mimir && cd mimir && melos clean && melos bootstrap && melos run test
Cloning into 'mimir'...
remote: Enumerating objects: 2609, done.
remote: Counting objects: 100% (576/576), done.
remote: Compressing objects: 100% (318/318), done.
remote: Total 2609 (delta 350), reused 322 (delta 257), pack-reused 2033
Receiving objects: 100% (2609/2609), 3.42 MiB | 10.97 MiB/s, done.
Resolving deltas: 100% (1364/1364), done.
Cleaning workspace...

Workspace cleaned. You will need to run the bootstrap command again to use this workspace.
melos bootstrap
  └> /home/atom/mimir

Running "flutter pub get" in workspace packages...
  ✓ mimir
    └> packages/mimir
  ✓ example
    └> packages/mimir/example
  ✓ flutter_mimir
    └> packages/flutter_mimir
  ✓ flutter_example
    └> packages/flutter_mimir/example
  > SUCCESS

Generating IntelliJ IDE files...
  > SUCCESS

 -> 4 packages bootstrapped
melos run test
  └> melos run test:dart --no-select && melos run test:flutter --no-select
     └> RUNNING

melos run test:dart
  └> melos exec -c 1 --fail-fast -- "dart test test"
     └> RUNNING

$ melos exec
  └> dart test test
     └> RUNNING (in 2 packages)

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
example:
00:01 +0 -1: test/example_test.dart: Main function runs correctly [E]
  Invalid argument(s): Failed to load dynamic library '../../../target/release/libembedded_milli.so': ../../../target/release/libembedded_milli.so: cannot open shared object file: No such file or directory
  dart:ffi                             new DynamicLibrary.open
  package:example/example.dart 136:25  getLibrary
  package:example/example.dart 16:28   main

To run this test again: /home/atom/snap/flutter/common/flutter/bin/cache/dart-sdk/bin/dart test test/example_test.dart -p vm --plain-name 'Main function runs correctly'
00:01 +0 -1: Some tests failed.

Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
For example, 'dart test --chain-stack-traces'.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

$ melos exec
  └> dart test test
     └> FAILED (in 1 packages)
        └> example (with exit code 1)

melos run test:dart
  └> melos exec -c 1 --fail-fast -- "dart test test"
     └> FAILED
ScriptException: The script test:dart failed to execute

melos run test
  └> melos run test:dart --no-select && melos run test:flutter --no-select
     └> FAILED
ScriptException: The script test failed to execute

As you can see, this test fails because the shared library cannot be found.

According to the guide you wrote, the shared library binaries are distributed through GitHub Releases, so I did not build them before running the tests. If I do build the shared library however, the result is similar:

Click to expand ``` atom@Dell-Desktop:~$ git clone https://github.com/GregoryConrad/mimir && cd mimir && melos clean && melos bootstrap && bash scripts/build-other.sh && melos run test Cloning into 'mimir'... remote: Enumerating objects: 2609, done. remote: Counting objects: 100% (542/542), done. remote: Compressing objects: 100% (311/311), done. remote: Total 2609 (delta 315), reused 295 (delta 230), pack-reused 2067 Receiving objects: 100% (2609/2609), 3.42 MiB | 11.60 MiB/s, done. Resolving deltas: 100% (1358/1358), done. Cleaning workspace... Workspace cleaned. You will need to run the bootstrap command again to use this workspace. melos bootstrap └> /home/atom/mimir Running "flutter pub get" in workspace packages... ✓ example └> packages/mimir/example ✓ mimir └> packages/mimir ✓ flutter_mimir └> packages/flutter_mimir ✓ flutter_example └> packages/flutter_mimir/example > SUCCESS Generating IntelliJ IDE files... > SUCCESS -> 4 packages bootstrapped Updating crates.io index Ignored package `cargo-zigbuild v0.14.2` is already installed, use --force to override Updating crates.io index Ignored package `cargo-xwin v0.13.3` is already installed, use --force to override info: component 'rust-std' for target 'aarch64-unknown-linux-gnu' is up to date Updating crates.io index Updating git repository `https://github.com/GregoryConrad/milli` Updating git repository `https://github.com/meilisearch/heed` Updating git repository `https://github.com/meilisearch/lmdb-rs` Compiling proc-macro2 v1.0.49 Compiling quote v1.0.23 Compiling unicode-ident v1.0.6 ... SNIP ... Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 2m 11s info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date Compiling libc v0.2.139 Compiling cfg-if v1.0.0 Compiling serde v1.0.151 ... SNIP ... Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 1m 22s info: component 'rust-std' for target 'aarch64-pc-windows-msvc' is up to date Compiling cfg-if v1.0.0 Compiling serde v1.0.151 Compiling winapi v0.3.9 ... SNIP ... Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 1m 27s info: component 'rust-std' for target 'x86_64-pc-windows-msvc' is up to date Compiling cfg-if v1.0.0 Compiling serde v1.0.151 Compiling winapi v0.3.9 ... SNIP ... Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 1m 27s linux-arm64/ linux-arm64/libembedded_milli.so linux-x64/ linux-x64/libembedded_milli.so windows-arm64/ windows-arm64/embedded_milli.dll windows-x64/ windows-x64/embedded_milli.dll melos run test └> melos run test:dart --no-select && melos run test:flutter --no-select └> RUNNING melos run test:dart └> melos exec -c 1 --fail-fast -- "dart test test" └> RUNNING $ melos exec └> dart test test └> RUNNING (in 2 packages) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- example: 00:01 +0 -1: test/example_test.dart: Main function runs correctly [E] Invalid argument(s): Failed to load dynamic library '../../../target/release/libembedded_milli.so': ../../../target/release/libembedded_milli.so: cannot open shared object file: No such file or directory dart:ffi new DynamicLibrary.open package:example/example.dart 136:25 getLibrary package:example/example.dart 16:28 main To run this test again: /home/atom/snap/flutter/common/flutter/bin/cache/dart-sdk/bin/dart test test/example_test.dart -p vm --plain-name 'Main function runs correctly' 00:02 +0 -1: Some tests failed. Consider enabling the flag chain-stack-traces to receive more detailed exceptions. For example, 'dart test --chain-stack-traces'. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- $ melos exec └> dart test test └> FAILED (in 1 packages) └> example (with exit code 1) melos run test:dart └> melos exec -c 1 --fail-fast -- "dart test test" └> FAILED ScriptException: The script test:dart failed to execute melos run test └> melos run test:dart --no-select && melos run test:flutter --no-select └> FAILED ScriptException: The script test failed to execute ```

I would appreciate it if you could give any advice on any debugging steps I could take to set up my system to run passing tests from this repository.

System information

Click to expand **I am running Ubuntu Linux via WSL.** #### Windows ``` C:\Users\Atom>ver Microsoft Windows [Version 10.0.19045.2364] ``` #### WSL ``` C:\Users\Atom>wsl --version WSL version: 1.0.3.0 Kernel version: 5.15.79.1 WSLg version: 1.0.47 MSRDC version: 1.2.3575 Direct3D version: 1.606.4 DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp Windows version: 10.0.19045.2364 ``` #### Linux Kernel ``` atom@Dell-Desktop:~$ uname -a Linux Dell-Desktop 5.15.79.1-microsoft-standard-WSL2 #1 SMP Wed Nov 23 01:01:46 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux ``` #### Flutter ``` atom@Dell-Desktop:~$ flutter doctor -v [✓] Flutter (Channel stable, 3.3.10, on Ubuntu 22.04.1 LTS 5.15.79.1-microsoft-standard-WSL2, locale C.UTF-8) • Flutter version 3.3.10 on channel stable at /home/atom/snap/flutter/common/flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision 135454af32 (9 days ago), 2022-12-15 07:36:55 -0800 • Engine revision 3316dd8728 • Dart version 2.18.6 • DevTools version 2.15.0 [✗] Android toolchain - develop for Android devices ✗ Unable to locate Android SDK. Install Android Studio from: https://developer.android.com/studio/index.html On first launch it will assist you in installing the Android SDK components. (or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions). If the Android SDK has been installed to a custom location, please use `flutter config --android-sdk` to update to that location. [✗] Chrome - develop for the web (Cannot find Chrome executable at google-chrome) ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable. [✓] Linux toolchain - develop for Linux desktop • clang version 10.0.0-4ubuntu1 • cmake version 3.16.3 • ninja version 1.10.0 • pkg-config version 0.29.1 [!] Android Studio (not installed) • Android Studio not found; download from https://developer.android.com/studio/index.html (or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions). [✓] Connected device (1 available) • Linux (desktop) • linux • linux-x64 • Ubuntu 22.04.1 LTS 5.15.79.1-microsoft-standard-WSL2 [✓] HTTP Host Availability • All required HTTP hosts are available ! Doctor found issues in 3 categories. ``` #### Flutter Snap (this is how Flutter and Dart are installed on my system) ``` atom@Dell-Desktop:~$ snap info flutter name: flutter summary: Flutter SDK publisher: Flutter Team✓ store-url: https://snapcraft.io/flutter contact: https://github.com/flutter/flutter/issues license: unset description: | Flutter is Google’s UI toolkit for building beautiful, natively compiled applications for mobile, web, and desktop from a single codebase. commands: - flutter.dart - flutter - flutter.openurl snap-id: YO0Adf4fqgxL7i3SaCn00oxEfd5CNQ63 tracking: latest/stable refresh-date: yesterday at 17:27 GMT channels: latest/stable: 0+git.ccbc698 2022-10-20 (130) 215MB classic latest/candidate: ↑ latest/beta: ↑ latest/edge: 0+git.74565fa 2022-11-29 (133) 216MB classic installed: 0+git.ccbc698 (130) 215MB classic ``` #### Rust ``` atom@Dell-Desktop:~$ rustup --version rustup 1.25.1 (bb60b1e89 2022-07-12) info: This is the version for the rustup toolchain manager, not the rustc compiler. info: The currently active `rustc` version is `rustc 1.66.0 (69f9c33d7 2022-12-12)` ```
GregoryConrad commented 1 year ago

Hi! Thanks for brining this up. I probably should've clarified that part in the guide. There are two things to note here:

To run Dart-only tests, always run cargo build -r beforehand. This is your issue here; you are missing the binaries locally. If you look in my Dart unit test source code, you will see that it references the binaries in the /target/release folder (or something like that), which doesn't exist (or might be out of date) unless you run cargo build -r right before running the Dart tests.

To run Flutter-specific tests, you will need to create the platform specific binaries, which is what you are mentioning. That can be done through melos run build (for all platform binaries, only possible on macOS, and takes ages to finish) or melos run build:other (windows/linux) or melos run build:android (android). Please note that after you run that build command, you will need to copy the binaries over to the correct part of your /packages/flutter_library_name directory. That is detailed here.

GregoryConrad commented 1 year ago

I will admit the current build system is somewhat of a pain right now, but unfortunately you can't do too much better until Dart supports Native Assets.

CicadaCinema commented 1 year ago

To run Dart-only tests, always run cargo build -r beforehand. This is your issue here; you are missing the binaries locally. If you look in my Dart unit test source code, you will see that it references the binaries in the /target/release folder (or something like that), which doesn't exist (or might be out of date) unless you run cargo build -r right before running the Dart tests.

Perfect, this works for me!

Please note that after you run that build command, you will need to copy the binaries over to the correct part of your /packages/flutter_library_name directory.

Could you explain why builds need to be run locally in this case? In the guide, you write:

When the Dart tooling builds our library (such as when an application consuming our library is built), it invokes the platform specific build process. We hijack this build process by downloading a copy of the binaries for the needed platform, if not already present on the filesystem. This last part is the key; it allows us to run integration tests locally and in CI by providing our own copy of the binaries instead of forcing our build process to always fetch the binaries from GitHub releases.

So only projects which depend on the Flutter library can take advantage of this? For example, if I wrote an example which depends on package:flutter_mimir and then wrote some tests for that example, would I not have to worry about platform-specific code or compiling these binaries at all?

GregoryConrad commented 1 year ago

Could you explain why builds need to be run locally in this case?

Sure. That is because you want any consumers of the library to automatically fetch the corresponding release binaries from online (GitHub releases). However, if you are developing locally and make changes on the Rust side, that behavior is incorrect; you would instead want to use the binaries you built locally instead of fetching them from online. The CMake & podspec modifications detailed in the guide only download binaries if not already present, which takes care of this behavior.

So only projects which depend on the Flutter library can take advantage of this? For example, if I wrote an example which depends on package:flutter_mimir and then wrote some tests for that example, would I not have to worry about platform-specific code or compiling these binaries at all?

Correct, by having the CMake/podspec scripts auto-download the binaries if they don't find them on the filesystem in the correct places, users of the library don't have to worry at all about it.

CicadaCinema commented 1 year ago

Great - thanks for clarifying!

It looks like the tests run fine after I run cargo build -r and melos run build:other, and then copy the generated archive to the relevant folder.

By the way, it looks like that build script has a little hiccup towards the end - I think mkdir runs on an existing directory

https://github.com/GregoryConrad/mimir/blob/597d3b0cf56037f4833dca7ce5d6c7fbe82aba5c/scripts/build-other.sh#L18-L19

which leads to the warnings towards the end of the following log:

Click to expand ``` atom@Dell-Desktop:~$ git clone https://github.com/GregoryConrad/mimir && cd mimir && melos clean && melos bootstrap && melos run build:other Cloning into 'mimir'... remote: Enumerating objects: 2609, done. remote: Counting objects: 100% (542/542), done. remote: Compressing objects: 100% (311/311), done. remote: Total 2609 (delta 315), reused 295 (delta 230), pack-reused 2067 Receiving objects: 100% (2609/2609), 3.42 MiB | 781.00 KiB/s, done. Resolving deltas: 100% (1358/1358), done. Cleaning workspace... Workspace cleaned. You will need to run the bootstrap command again to use this workspace. melos bootstrap └> /home/atom/mimir Running "flutter pub get" in workspace packages... ✓ mimir └> packages/mimir ✓ example └> packages/mimir/example [flutter_mimir]: Running "flutter pub get" in flutter_mimir... [flutter_mimir]: Running "flutter pub get" in flutter_mimir... 3.5s [flutter_mimir]: Running "flutter pub get" in example... [flutter_mimir]: Running "flutter pub get" in example... 3.4s ✓ flutter_mimir └> packages/flutter_mimir ✓ flutter_example └> packages/flutter_mimir/example > SUCCESS Generating IntelliJ IDE files... > SUCCESS -> 4 packages bootstrapped melos run build:other └> bash scripts/build-other.sh └> RUNNING Updating crates.io index Ignored package `cargo-zigbuild v0.14.2` is already installed, use --force to override Updating crates.io index Ignored package `cargo-xwin v0.13.3` is already installed, use --force to override info: component 'rust-std' for target 'aarch64-unknown-linux-gnu' is up to date Updating crates.io index Updating git repository `https://github.com/GregoryConrad/milli` Updating git repository `https://github.com/meilisearch/heed` Updating git repository `https://github.com/meilisearch/lmdb-rs` Compiling proc-macro2 v1.0.49 Compiling quote v1.0.23 Compiling unicode-ident v1.0.6 ... SNIP ... Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 4m 11s info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date Compiling libc v0.2.139 Compiling cfg-if v1.0.0 Compiling serde v1.0.151 ... SNIP ... Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 2m 49s info: component 'rust-std' for target 'aarch64-pc-windows-msvc' is up to date Error: File exists (os error 17) cp: cannot stat '../target/aarch64-pc-windows-msvc/release/embedded_milli.dll': No such file or directory info: component 'rust-std' for target 'x86_64-pc-windows-msvc' is up to date Error: File exists (os error 17) cp: cannot stat '../target/x86_64-pc-windows-msvc/release/embedded_milli.dll': No such file or directory linux-arm64/ linux-arm64/libembedded_milli.so linux-x64/ linux-x64/libembedded_milli.so windows-arm64/ windows-x64/ melos run build:other └> bash scripts/build-other.sh └> SUCCESS ```
GregoryConrad commented 1 year ago

If memory serves, that mkdir failing shouldn't be an issue. If the directory is already created, it doesn't need to be created again (and can just fail). However, the:

cp: cannot stat '../target/x86_64-pc-windows-msvc/release/embedded_milli.dll': No such file or directory

Does look like an issue. For some reason the .dll is not being created; would you mind pasting the entire output here? Not sure what is going on there.

GregoryConrad commented 1 year ago

Or perhaps the .dll is in a different location that I expected. Maybe it is created in a different spot when running on WSL?

CicadaCinema commented 1 year ago

would you mind pasting the entire output here? Not sure what is going on there.

The snipped text is just repeated "Compiling" logs:

Click to expand ``` atom@Dell-Desktop:~$ git clone https://github.com/GregoryConrad/mimir && cd mimir && melos clean && melos bootstrap && melos run build:other Cloning into 'mimir'... remote: Enumerating objects: 2617, done. remote: Counting objects: 100% (580/580), done. remote: Compressing objects: 100% (324/324), done. remote: Total 2617 (delta 350), reused 322 (delta 255), pack-reused 2037 Receiving objects: 100% (2617/2617), 3.42 MiB | 12.66 MiB/s, done. Resolving deltas: 100% (1367/1367), done. Cleaning workspace... Workspace cleaned. You will need to run the bootstrap command again to use this workspace. melos bootstrap └> /home/atom/mimir Running "flutter pub get" in workspace packages... ✓ mimir └> packages/mimir ✓ example └> packages/mimir/example ✓ flutter_mimir └> packages/flutter_mimir ✓ flutter_example └> packages/flutter_mimir/example > SUCCESS Generating IntelliJ IDE files... > SUCCESS -> 4 packages bootstrapped melos run build:other └> bash scripts/build-other.sh └> RUNNING Updating crates.io index Ignored package `cargo-zigbuild v0.14.2` is already installed, use --force to override Updating crates.io index Ignored package `cargo-xwin v0.13.3` is already installed, use --force to override info: component 'rust-std' for target 'aarch64-unknown-linux-gnu' is up to date Updating crates.io index Updating git repository `https://github.com/GregoryConrad/milli` Updating git repository `https://github.com/meilisearch/heed` Updating git repository `https://github.com/meilisearch/lmdb-rs` Compiling proc-macro2 v1.0.49 Compiling quote v1.0.23 Compiling unicode-ident v1.0.6 Compiling syn v1.0.107 Compiling autocfg v1.1.0 Compiling serde_derive v1.0.151 Compiling serde v1.0.151 Compiling libc v0.2.139 Compiling memchr v2.5.0 Compiling indexmap v1.9.2 Compiling cfg-if v1.0.0 Compiling version_check v0.9.4 Compiling log v0.4.17 Compiling anyhow v1.0.68 Compiling proc-macro-error-attr v1.0.4 Compiling byteorder v1.4.3 Compiling proc-macro-error v1.0.4 Compiling thiserror v1.0.38 Compiling build_const v0.2.2 Compiling crc v1.8.1 Compiling heck v0.4.0 Compiling hashbrown v0.12.3 Compiling encoding_index_tests v0.1.4 Compiling serde_json v1.0.91 Compiling atty v0.2.14 Compiling os_str_bytes v6.4.1 Compiling termcolor v1.1.3 Compiling once_cell v1.16.0 Compiling clap_lex v0.2.4 Compiling aho-corasick v0.7.20 Compiling bitflags v1.3.2 Compiling regex-syntax v0.6.28 Compiling ryu v1.0.12 Compiling textwrap v0.16.0 Compiling semver v1.0.16 Compiling strsim v0.10.0 Compiling encoding-index-simpchinese v1.20141219.5 Compiling encoding-index-singlebyte v1.20141219.5 Compiling encoding-index-korean v1.20141219.5 Compiling encoding-index-japanese v1.20141219.5 Compiling encoding-index-tradchinese v1.20141219.5 Compiling humantime v2.1.0 Compiling crossbeam-utils v0.8.14 Compiling encoding v0.2.33 Compiling yada v0.5.0 Compiling regex v1.7.0 Compiling lazy_static v1.4.0 Compiling libm v0.2.6 Compiling thiserror-impl v1.0.38 Compiling clap_derive v3.2.18 Compiling env_logger v0.9.3 Compiling fst v0.4.7 Compiling num-traits v0.2.15 Compiling crc32fast v1.3.2 Compiling clap v3.2.23 Compiling itoa v1.0.5 Compiling smallvec v1.10.0 Compiling regex-automata v0.1.10 Compiling bincode v1.3.3 Compiling lindera-core v0.13.5 Compiling lzma-rs v0.2.0 Compiling siphasher v0.3.10 Compiling adler v1.0.2 Compiling rand_core v0.6.4 Compiling cc v1.0.78 Compiling scopeguard v1.1.0 Compiling rand v0.8.5 Compiling miniz_oxide v0.6.2 Compiling phf_shared v0.11.1 Compiling lindera-decompress v0.13.5 Compiling bstr v0.2.17 Compiling filetime v0.2.19 Compiling xattr v0.2.3 Compiling csv-core v0.1.10 Compiling memoffset v0.7.1 Compiling itoa v0.4.8 Compiling glob v0.3.0 Compiling csv v1.1.6 Compiling lindera-ipadic-builder v0.13.5 Compiling tar v0.4.38 Compiling flate2 v1.0.25 Compiling phf_generator v0.11.1 Compiling crossbeam-epoch v0.9.13 Compiling camino v1.1.1 Compiling unicode-xid v0.2.4 Compiling synstructure v0.12.6 Compiling lindera-ipadic v0.13.5 Compiling phf_codegen v0.11.1 Compiling rustc_version v0.4.0 Compiling bytemuck_derive v1.3.0 Compiling num_cpus v1.15.0 Compiling hashbrown v0.7.2 Compiling num-integer v0.1.45 Compiling linked-hash-map v0.5.6 Compiling fastrand v1.8.0 Compiling pkg-config v0.3.26 Compiling cbindgen v0.24.3 Compiling crossbeam-queue v0.3.8 Compiling remove_dir_all v0.5.3 Compiling tempfile v3.3.0 Compiling lmdb-rkv-sys v0.15.1 (https://github.com/meilisearch/lmdb-rs#0144fb2b) Compiling yaml-rust v0.4.5 Compiling bytemuck v1.12.3 Compiling character_converter v2.1.4 Compiling heapless v0.7.16 Compiling jieba-rs v0.6.7 Compiling zerocopy-derive v0.2.0 Compiling toml v0.5.10 Compiling cargo-platform v0.1.2 Compiling lock_api v0.4.9 Compiling atomic v0.5.1 Compiling minimal-lexical v0.2.1 Compiling either v1.8.0 Compiling rayon-core v1.10.1 Compiling ahash v0.3.8 Compiling parking_lot_core v0.9.5 Compiling nom v7.1.1 Compiling itertools v0.10.5 Compiling zerocopy v0.3.0 Compiling cargo_metadata v0.14.2 Compiling crossbeam-deque v0.8.2 Compiling serde_yaml v0.8.26 Compiling phf v0.11.1 Compiling lindera-unidic-builder v0.13.5 Compiling lindera-cc-cedict-builder v0.13.5 Compiling lindera-ko-dic-builder v0.13.5 Compiling lindera-dictionary v0.13.5 Compiling cedarwood v0.4.6 Compiling crossbeam-channel v0.5.6 Compiling enum_dispatch v0.3.8 Compiling extend v1.1.2 Compiling delegate v0.8.0 Compiling time v0.1.45 Compiling fxhash v0.2.1 Compiling hash32 v0.2.1 Compiling smartstring v1.0.1 Compiling iana-time-zone v0.1.53 Compiling convert_case v0.5.0 Compiling bytecount v0.6.3 Compiling heed-traits v0.7.0 (https://github.com/meilisearch/heed?tag=v0.12.4#7a4542bc) Compiling build-target v0.4.0 Compiling time-core v0.1.0 Compiling pathdiff v0.2.1 Compiling stable_deref_trait v1.2.0 Compiling flutter_rust_bridge_codegen v1.57.0 Compiling time-macros v0.2.6 Compiling flutter_rust_bridge v1.57.0 Compiling heed-types v0.7.2 (https://github.com/meilisearch/heed?tag=v0.12.4#7a4542bc) Compiling nom_locate v4.0.0 Compiling chrono v0.4.23 Compiling synchronoise v1.0.1 Compiling whatlang v0.13.0 Compiling logging_timer_proc_macros v1.1.0 Compiling page_size v0.4.2 Compiling getrandom v0.2.8 Compiling retain_mut v0.1.7 Compiling unicode-segmentation v1.10.0 Compiling deunicode v1.3.3 Compiling static_assertions v1.1.0 Compiling cow-utils v0.1.2 Compiling slice-group-by v0.3.0 Compiling logging_timer v1.1.0 Compiling time v0.3.17 Compiling embedded_milli v0.0.0 (/home/atom/mimir/packages/mimir/native) Compiling roaring v0.10.1 Compiling uuid v1.2.2 Compiling heed v0.12.4 (https://github.com/meilisearch/heed?tag=v0.12.4#7a4542bc) Compiling parking_lot v0.12.1 Compiling allo-isolate v0.1.14-beta.4 Compiling rayon v1.6.1 Compiling grenad v0.4.4 Compiling filter-parser v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Compiling rstar v0.9.3 Compiling levenshtein_automata v0.2.1 Compiling threadpool v1.8.1 Compiling flatten-serde-json v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Compiling json-depth-checker v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Compiling ordered-float v3.4.0 Compiling bimap v0.6.2 Compiling smallstr v0.3.0 Compiling bstr v1.1.0 Compiling concat-arrays v0.1.2 Compiling memmap2 v0.5.8 Compiling flutter_rust_bridge_macros v1.57.0 Compiling geoutils v0.5.1 Compiling obkv v0.2.0 Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 2m 18s info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date Compiling libc v0.2.139 Compiling cfg-if v1.0.0 Compiling serde v1.0.151 Compiling byteorder v1.4.3 Compiling memchr v2.5.0 Compiling encoding_index_tests v0.1.4 Compiling hashbrown v0.12.3 Compiling anyhow v1.0.68 Compiling log v0.4.17 Compiling indexmap v1.9.2 Compiling once_cell v1.16.0 Compiling ryu v1.0.12 Compiling encoding-index-singlebyte v1.20141219.5 Compiling encoding-index-simpchinese v1.20141219.5 Compiling encoding-index-korean v1.20141219.5 Compiling encoding-index-tradchinese v1.20141219.5 Compiling encoding-index-japanese v1.20141219.5 Compiling thiserror v1.0.38 Compiling scopeguard v1.1.0 Compiling encoding v0.2.33 Compiling crossbeam-utils v0.8.14 Compiling aho-corasick v0.7.20 Compiling crc v1.8.1 Compiling yada v0.5.0 Compiling lazy_static v1.4.0 Compiling regex-syntax v0.6.28 Compiling libm v0.2.6 Compiling atty v0.2.14 Compiling smallvec v1.10.0 Compiling bincode v1.3.3 Compiling os_str_bytes v6.4.1 Compiling regex v1.7.0 Compiling termcolor v1.1.3 Compiling lindera-core v0.13.5 Compiling regex-automata v0.1.10 Compiling itoa v1.0.5 Compiling clap_lex v0.2.4 Compiling num-traits v0.2.15 Compiling lzma-rs v0.2.0 Compiling serde_json v1.0.91 Compiling bitflags v1.3.2 Compiling humantime v2.1.0 Compiling textwrap v0.16.0 Compiling strsim v0.10.0 Compiling clap v3.2.23 Compiling lindera-decompress v0.13.5 Compiling env_logger v0.9.3 Compiling lock_api v0.4.9 Compiling bstr v0.2.17 Compiling csv-core v0.1.10 Compiling itoa v0.4.8 Compiling glob v0.3.0 Compiling csv v1.1.6 Compiling memoffset v0.7.1 Compiling crossbeam-epoch v0.9.13 Compiling fst v0.4.7 Compiling num_cpus v1.15.0 Compiling siphasher v0.3.10 Compiling crossbeam-queue v0.3.8 Compiling num-integer v0.1.45 Compiling phf_shared v0.11.1 Compiling hashbrown v0.7.2 Compiling bytemuck v1.12.3 Compiling lindera-ipadic v0.13.5 Compiling ahash v0.3.8 Compiling minimal-lexical v0.2.1 Compiling rayon-core v1.10.1 Compiling parking_lot_core v0.9.5 Compiling nom v7.1.1 Compiling crossbeam-deque v0.8.2 Compiling atomic v0.5.1 Compiling zerocopy v0.3.0 Compiling character_converter v2.1.4 Compiling jieba-rs v0.6.7 Compiling heapless v0.7.16 Compiling phf v0.11.1 Compiling lmdb-rkv-sys v0.15.1 (https://github.com/meilisearch/lmdb-rs#0144fb2b) Compiling lindera-cc-cedict-builder v0.13.5 Compiling lindera-ko-dic-builder v0.13.5 Compiling lindera-unidic-builder v0.13.5 Compiling spin v0.9.4 Compiling lindera-ipadic-builder v0.13.5 Compiling lindera-dictionary v0.13.5 Compiling cedarwood v0.4.6 Compiling crossbeam-channel v0.5.6 Compiling time v0.1.45 Compiling fxhash v0.2.1 Compiling hash32 v0.2.1 Compiling either v1.8.0 Compiling fastrand v1.8.0 Compiling stable_deref_trait v1.2.0 Compiling iana-time-zone v0.1.53 Compiling remove_dir_all v0.5.3 Compiling heed-traits v0.7.0 (https://github.com/meilisearch/heed?tag=v0.12.4#7a4542bc) Compiling bytecount v0.6.3 Compiling heed-types v0.7.2 (https://github.com/meilisearch/heed?tag=v0.12.4#7a4542bc) Compiling tempfile v3.3.0 Compiling nom_locate v4.0.0 Compiling chrono v0.4.23 Compiling smartstring v1.0.1 Compiling synchronoise v1.0.1 Compiling whatlang v0.13.0 Compiling getrandom v0.2.8 Compiling page_size v0.4.2 Compiling unicode-segmentation v1.10.0 Compiling deunicode v1.3.3 Compiling retain_mut v0.1.7 Compiling slice-group-by v0.3.0 Compiling time-core v0.1.0 Compiling static_assertions v1.1.0 Compiling cow-utils v0.1.2 Compiling time v0.3.17 Compiling roaring v0.10.1 Compiling logging_timer v1.1.0 Compiling heed v0.12.4 (https://github.com/meilisearch/heed?tag=v0.12.4#7a4542bc) Compiling uuid v1.2.2 Compiling parking_lot v0.12.1 Compiling allo-isolate v0.1.14-beta.4 Compiling rayon v1.6.1 Compiling rstar v0.9.3 Compiling filter-parser v 0 . 37 . 0 ( https://github.com/GregoryConrad/milli ? branch= filter-parser-convenience # 50954d31 ) Compiling grenad v0.4.4 Compiling flutter_rust_bridge v1.57.0 Compiling itertools v0.10.5 Compiling levenshtein_automata v0.2.1 Compiling threadpool v1.8.1 Compiling flatten-serde-json v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Compiling json-depth-checker v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Compiling ordered-float v3.4.0 Compiling bstr v1.1.0 Compiling bimap v0.6.2 Compiling smallstr v0.3.0 Compiling memmap2 v0.5.8 Compiling obkv v0.2.0 Compiling geoutils v0.5.1 Compiling embedded_milli v0.0.0 (/home/atom/mimir/packages/mimir/native) Compiling lindera v0.14.0 Compiling charabia v0.6.0 Compiling milli v0.37.0 (https://github.com/GregoryConrad/milli?branch=filter-parser-convenience#50954d31) Finished release [optimized] target(s) in 1m 34s info: component 'rust-std' for target 'aarch64-pc-windows-msvc' is up to date Error: File exists (os error 17) cp: cannot stat '../target/aarch64-pc-windows-msvc/release/embedded_milli.dll': No such file or directory info: component 'rust-std' for target 'x86_64-pc-windows-msvc' is up to date Error: File exists (os error 17) cp: cannot stat '../target/x86_64-pc-windows-msvc/release/embedded_milli.dll': No such file or directory linux-arm64/ linux-arm64/libembedded_milli.so linux-x64/ linux-x64/libembedded_milli.so windows-arm64/ windows-x64/ melos run build:other └> bash scripts/build-other.sh └> SUCCESS atom@Dell-Desktop:~/mimir$ ```

It looks like no Windows libraries are present at all:

atom@Dell-Desktop:~/mimir$ tree -L 2 target
target
├── CACHEDIR.TAG
├── aarch64-unknown-linux-gnu
│   ├── CACHEDIR.TAG
│   └── release
├── release
│   ├── build
│   ├── deps
│   ├── examples
│   └── incremental
└── x86_64-unknown-linux-gnu
    ├── CACHEDIR.TAG
    └── release

9 directories, 3 files

In fact, it was cargo-xwin which produced this 'os error 17'. Not sure what's going on here:

atom@Dell-Desktop:~/mimir$ rustup target add aarch64-pc-windows-msvc
info: component 'rust-std' for target 'aarch64-pc-windows-msvc' is up to date
atom@Dell-Desktop:~/mimir$ cargo xwin build --target aarch64-pc-windows-msvc -r
Error: File exists (os error 17)
atom@Dell-Desktop:~/mimir$ cargo xwin --version
cargo-xwin-xwin 0.13.3
GregoryConrad commented 1 year ago

Ahh! Thanks for looking into that. Assuming you have LLVM installed, probably an issue with cargo-xwin then. Seeing as everything needed by mimir builds in CI I probably won't look into it any further but if that is blocking for you, you may want to file an issue over at that project. But, seeing as you are developing on Windows, you could also just probably build the binaries locally and not have to interface thru cargo-xwin.

CicadaCinema commented 1 year ago

Yeah, I agree.

GregoryConrad commented 1 year ago

@CicadaCinema I just realized an issue when running in iOS/macOS in release mode. Binaries were getting stripped as-is. Going to file a PR to fix the docs I made in FRB but since you have already gone thru the guide, switch out your EnforceBundling.swift files with the following:

public func dummyMethodToEnforceBundling() -> Int64 {
  return dummy_method_to_enforce_bundling()
}
let dummyVar = dummyMethodToEnforceBundling();

Annoying that the previous version didn't work so you will need to change to that.

CicadaCinema commented 1 year ago

@CicadaCinema I just realized an issue when running in iOS/macOS in release mode. Binaries were getting stripped as-is. Going to file a PR to fix the docs I made in FRB but since you have already gone thru the guide, switch out your EnforceBundling.swift files with the following:

public func dummyMethodToEnforceBundling() -> Int64 {
  return dummy_method_to_enforce_bundling()
}
let dummyVar = dummyMethodToEnforceBundling();

Annoying that the previous version didn't work so you will need to change to that.

This is unrelated to the issue with cargo-xwin, right?

Thanks for the heads-up! For now I think I'll stick with targeting Linux and Windows though - at least until Native Assets releases.

GregoryConrad commented 1 year ago

This is unrelated to the issue with cargo-xwin, right?

correct; just figured I’d let you know since it seemed as though you got through a substantial part of the guide already.

For now I think I'll stick with targeting Linux and Windows though - at least until Native Assets releases.

and that’s a good decision! It’s really a pain to deal with all the platform specific setup as-is.

CicadaCinema commented 1 year ago

@GregoryConrad Hi, sorry about bothering you again - I've attempted to write a simple template following your guide, and I'm running into similar issues again.

The corresponding repository is https://github.com/CicadaCinema/flutter_dart_rust_template and I've decided to use a slightly different naming convention to yours, to be able to more easily identify what's what and to be able to replace these strings when using this template in a real project: bbb_project_bbb is the pure-Dart project and ccc_project_ccc is the Flutter project which wraps it; these can both be found in packages/. aaa_project_aaa is the name of the native Rust library which can be found in packages/bbb_project_bbb/native/.

In this commit I have created a simple adder function and exported it through bbb_project_bbb and ccc_project_ccc. I have also added some Dart tests for these wrappers in the same commit. After building the native library (and moving/renaming the resulting archive to the relevant folder - see logs), the Dart test passes, but the Flutter test (in packages/ccc_project_ccc/) fails!

Do you think you could take a look?

Logs

Click to expand ``` atom@Dell-Desktop:~$ git clone https://github.com/CicadaCinema/flutter_dart_rust_template && cd flutter_dart_rust_template && melos clean && melos bootstrap Cloning into 'flutter_dart_rust_template'... remote: Enumerating objects: 141, done. remote: Counting objects: 100% (141/141), done. remote: Compressing objects: 100% (98/98), done. remote: Total 141 (delta 39), reused 136 (delta 37), pack-reused 0 Receiving objects: 100% (141/141), 55.22 KiB | 2.76 MiB/s, done. Resolving deltas: 100% (39/39), done. Cleaning workspace... Workspace cleaned. You will need to run the bootstrap command again to use this workspace. melos bootstrap └> /home/atom/flutter_dart_rust_template Running "flutter pub get" in workspace packages... ✓ bbb_project_bbb └> packages/bbb_project_bbb ✓ ccc_project_ccc └> packages/ccc_project_ccc ✓ ccc_project_ccc_example └> packages/ccc_project_ccc/example Linking workspace packages... > SUCCESS Generating IntelliJ IDE files... > SUCCESS -> 3 packages bootstrapped atom@Dell-Desktop:~/flutter_dart_rust_template$ melos run build:other melos run build:other └> bash scripts/build-other.sh └> RUNNING Updating crates.io index Ignored package `cargo-zigbuild v0.14.2` is already installed, use --force to override Updating crates.io index Ignored package `cargo-xwin v0.13.3` is already installed, use --force to override info: component 'rust-std' for target 'aarch64-unknown-linux-gnu' is up to date Updating crates.io index Compiling proc-macro2 v1.0.49 Compiling quote v1.0.23 Compiling unicode-ident v1.0.6 Compiling autocfg v1.1.0 Compiling syn v1.0.107 Compiling libc v0.2.139 Compiling serde_derive v1.0.151 Compiling version_check v0.9.4 Compiling serde v1.0.151 Compiling proc-macro-error-attr v1.0.4 Compiling log v0.4.17 Compiling proc-macro-error v1.0.4 Compiling anyhow v1.0.68 Compiling indexmap v1.9.2 Compiling memchr v2.5.0 Compiling serde_json v1.0.91 Compiling ryu v1.0.12 Compiling hashbrown v0.12.3 Compiling cfg-if v1.0.0 Compiling atty v0.2.14 Compiling num-traits v0.2.15 Compiling os_str_bytes v6.4.1 Compiling itoa v1.0.5 Compiling camino v1.1.1 Compiling termcolor v1.1.3 Compiling once_cell v1.16.0 Compiling heck v0.4.0 Compiling semver v1.0.16 Compiling aho-corasick v0.7.20 Compiling clap_lex v0.2.4 Compiling num-integer v0.1.45 Compiling fastrand v1.8.0 Compiling remove_dir_all v0.5.3 Compiling thiserror v1.0.38 Compiling cbindgen v0.24.3 Compiling bitflags v1.3.2 Compiling linked-hash-map v0.5.6 Compiling textwrap v0.16.0 Compiling regex-syntax v0.6.28 Compiling strsim v0.10.0 Compiling yaml-rust v0.4.5 Compiling tempfile v3.3.0 Compiling lock_api v0.4.9 Compiling atomic v0.5.1 Compiling humantime v2.1.0 Compiling either v1.8.0 Compiling parking_lot_core v0.9.5 Compiling regex v1.7.0 Compiling itertools v0.10.5 Compiling env_logger v0.9.3 Compiling time v0.1.45 Compiling cc v1.0.78 Compiling pathdiff v0.2.1 Compiling build-target v0.4.0 Compiling clap_derive v3.2.18 Compiling thiserror-impl v1.0.38 Compiling delegate v0.8.0 Compiling extend v1.1.2 Compiling enum_dispatch v0.3.8 Compiling convert_case v0.5.0 Compiling scopeguard v1.1.0 Compiling smallvec v1.10.0 Compiling iana-time-zone v0.1.53 Compiling lazy_static v1.4.0 Compiling chrono v0.4.23 Compiling flutter_rust_bridge v1.57.0 Compiling num_cpus v1.15.0 Compiling clap v3.2.23 Compiling allo-isolate v0.1.14-beta.4 Compiling threadpool v1.8.1 Compiling parking_lot v0.12.1 Compiling flutter_rust_bridge_macros v1.57.0 Compiling bytemuck v1.12.3 Compiling cargo-platform v0.1.2 Compiling toml v0.5.10 Compiling serde_yaml v0.8.26 Compiling cargo_metadata v0.14.2 Compiling flutter_rust_bridge_codegen v1.57.0 Compiling aaa_project_aaa v0.0.0 (/home/atom/flutter_dart_rust_template/packages/bbb_project_bbb/native) Finished release [optimized] target(s) in 35.72s info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date Compiling libc v0.2.139 Compiling num-traits v0.2.15 Compiling num-integer v0.1.45 Compiling cfg-if v1.0.0 Compiling parking_lot_core v0.9.5 Compiling lock_api v0.4.9 Compiling atomic v0.5.1 Compiling anyhow v1.0.68 Compiling scopeguard v1.1.0 Compiling smallvec v1.10.0 Compiling iana-time-zone v0.1.53 Compiling log v0.4.17 Compiling flutter_rust_bridge v1.57.0 Compiling lazy_static v1.4.0 Compiling time v0.1.45 Compiling num_cpus v1.15.0 Compiling parking_lot v0.12.1 Compiling threadpool v1.8.1 Compiling chrono v0.4.23 Compiling bytemuck v1.12.3 Compiling aaa_project_aaa v0.0.0 (/home/atom/flutter_dart_rust_template/packages/bbb_project_bbb/native) Compiling allo-isolate v0.1.14-beta.4 Finished release [optimized] target(s) in 6.90s info: component 'rust-std' for target 'aarch64-pc-windows-msvc' is up to date Error: File exists (os error 17) cp: cannot stat '../target/aarch64-pc-windows-msvc/release/aaa_project_aaa.dll': No such file or directory info: component 'rust-std' for target 'x86_64-pc-windows-msvc' is up to date Error: File exists (os error 17) cp: cannot stat '../target/x86_64-pc-windows-msvc/release/aaa_project_aaa.dll': No such file or directory linux-arm64/ linux-arm64/libaaa_project_aaa.so linux-x64/ linux-x64/libaaa_project_aaa.so windows-arm64/ windows-x64/ melos run build:other └> bash scripts/build-other.sh └> SUCCESS atom@Dell-Desktop:~/flutter_dart_rust_template$ cargo build -r Compiling libc v0.2.139 Compiling num-traits v0.2.15 Compiling num-integer v0.1.45 Compiling cfg-if v1.0.0 Compiling parking_lot_core v0.9.5 Compiling atomic v0.5.1 Compiling lock_api v0.4.9 Compiling anyhow v1.0.68 Compiling iana-time-zone v0.1.53 Compiling smallvec v1.10.0 Compiling scopeguard v1.1.0 Compiling log v0.4.17 Compiling flutter_rust_bridge v1.57.0 Compiling bytemuck v1.12.3 Compiling time v0.1.45 Compiling num_cpus v1.15.0 Compiling lazy_static v1.4.0 Compiling threadpool v1.8.1 Compiling parking_lot v0.12.1 Compiling aaa_project_aaa v0.0.0 (/home/atom/flutter_dart_rust_template/packages/bbb_project_bbb/native) Compiling chrono v0.4.23 Compiling allo-isolate v0.1.14-beta.4 Finished release [optimized] target(s) in 8.59s atom@Dell-Desktop:~/flutter_dart_rust_template$ mv platform-build/other.tar.gz packages/ccc_project_ccc/linux/aaa_project_aaa-v0.0.0.tar.gz atom@Dell-Desktop:~/flutter_dart_rust_template$ ls -lah packages/ccc_project_ccc/linux total 2.3M drwxr-xr-x 2 atom atom 4.0K Dec 25 17:22 . drwxr-xr-x 11 atom atom 4.0K Dec 25 17:21 .. -rw-r--r-- 1 atom atom 1.3K Dec 25 17:21 CMakeLists.txt -rw-r--r-- 1 atom atom 2.3M Dec 25 17:22 aaa_project_aaa-v0.0.0.tar.gz atom@Dell-Desktop:~/flutter_dart_rust_template$ melos run test melos run test └> melos run test:dart --no-select && melos run test:flutter --no-select └> RUNNING melos run test:dart └> melos exec -c 1 --fail-fast -- "dart test test" └> RUNNING $ melos exec └> dart test test └> RUNNING (in 1 packages) ------------------------------------------------------------------------------------------------------------------------ bbb_project_bbb: 00:00 +0: loading test/bbb_project_bbb_test.dart 00:01 +0: loading test/bbb_project_bbb_test.dart 00:01 +0: test/bbb_project_bbb_test.dart: A group of tests First Test 00:01 +1: test/bbb_project_bbb_test.dart: A group of tests First Test 00:01 +1: test/bbb_project_bbb_test.dart: Tesing bbbAdd from package:bbb_project_bbb 00:01 +2: test/bbb_project_bbb_test.dart: Tesing bbbAdd from package:bbb_project_bbb 00:01 +2: All tests passed! bbb_project_bbb: SUCCESS ------------------------------------------------------------------------------------------------------------------------ $ melos exec └> dart test test └> SUCCESS melos run test:dart └> melos exec -c 1 --fail-fast -- "dart test test" └> SUCCESS melos run test:flutter └> melos exec -c 1 --fail-fast -- "flutter test test" └> RUNNING $ melos exec └> flutter test test └> RUNNING (in 2 packages) ------------------------------------------------------------------------------------------------------------------------ ccc_project_ccc: 00:00 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:01 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:02 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:03 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:04 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:05 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:06 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:07 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:08 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart 00:08 +0: Tesing cccAdd from package:ccc_project_ccc 00:08 +0 -1: Tesing cccAdd from package:ccc_project_ccc [E] Invalid argument(s): Failed to load dynamic library 'libaaa_project_aaa.so': libaaa_project_aaa.so: cannot open shared object file: No such file or directory dart:ffi new DynamicLibrary.open package:ccc_project_ccc/src/ffi/io.dart 12:27 createLibraryImpl package:ccc_project_ccc/src/ffi.dart 8:19 createLib package:ccc_project_ccc/ccc_project_ccc.dart 11:24 cccAdd test/ccc_project_ccc_test.dart 6:18 main. To run this test again: /home/atom/flutter/bin/cache/dart-sdk/bin/dart test /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/test/ccc_project_ccc_test.dart -p vm --plain-name 'Tesing cccAdd from package:ccc_project_ccc' 00:08 +0 -1: Some tests failed. ------------------------------------------------------------------------------------------------------------------------ $ melos exec └> flutter test test └> FAILED (in 1 packages) └> ccc_project_ccc (with exit code 1) melos run test:flutter └> melos exec -c 1 --fail-fast -- "flutter test test" └> FAILED ScriptException: The script test:flutter failed to execute melos run test └> melos run test:dart --no-select && melos run test:flutter --no-select └> FAILED ScriptException: The script test failed to execute ```
GregoryConrad commented 1 year ago

Invalid argument(s): Failed to load dynamic library 'libaaa_project_aaa.so': libaaa_project_aaa.so: cannot open shared object file: No such file or directory

Looks like your tests can't find the .so. Did you copy the .tag.gz archive to the correct place (/packages/bbb_project_bbb/linux/)?

GregoryConrad commented 1 year ago

I haven't tried running Flutter unit tests like that. It may be worth trying to run/use Flutter integration tests, assuming you have copied over the .tar.gz with the correct name based on your pubspec.yaml.

CicadaCinema commented 1 year ago

Did you copy the .tag.gz archive to the correct place (/packages/bbb_project_bbb/linux/)?

Yep! Excerpt from the logs:

atom@Dell-Desktop:~/flutter_dart_rust_template$ mv platform-build/other.tar.gz packages/ccc_project_ccc/linux/aaa_project_aaa-v0.0.0.tar.gz
atom@Dell-Desktop:~/flutter_dart_rust_template$ ls -lah packages/ccc_project_ccc/linux
total 2.3M
drwxr-xr-x  2 atom atom 4.0K Dec 25 17:22 .
drwxr-xr-x 11 atom atom 4.0K Dec 25 17:21 ..
-rw-r--r--  1 atom atom 1.3K Dec 25 17:21 CMakeLists.txt
-rw-r--r--  1 atom atom 2.3M Dec 25 17:22 aaa_project_aaa-v0.0.0.tar.gz
GregoryConrad commented 1 year ago

Based on your linux/CMakeLists.txt, looks like that should be working. I never tried doing Flutter unit tests, which is why I didn't write much on it in the guide. Looks like you probably won't be able to use Rust in Flutter unit tests, but integration tests should work.

GregoryConrad commented 1 year ago

Oh! You could also do something similar to the Dart-only unit tests for the Flutter unit tests, by referencing /target/release/...so! Forgot about that for a second.

GregoryConrad commented 1 year ago

I.e. you will need to get your Dynamic Library instance manually by using DynamicLibrary.open() and point it to your /target/release/... built file. You won't be able to rely upon the FFI convenience I made (createLib()), at least in Flutter unit tests.

CicadaCinema commented 1 year ago

Looks like you probably won't be able to use Rust in Flutter unit tests, but integration tests should work.

Is this true even for packages depending on ccc_project_ccc? I removed the unit tests in ccc_project_ccc, leaving the unit tests in ccc_project_ccc_example. These fail too:

Click to expand ``` atom@Dell-Desktop:~/flutter_dart_rust_template$ melos run test melos run test └> melos run test:dart --no-select && melos run test:flutter --no-select └> RUNNING melos run test:dart └> melos exec -c 1 --fail-fast -- "dart test test" └> RUNNING $ melos exec └> dart test test └> RUNNING (in 1 packages) ------------------------------------------------------------------------------------------------------------------------bbb_project_bbb: 00:00 +0: loading test/bbb_project_bbb_test.dart 00:01 +0: loading test/bbb_project_bbb_test.dart 00:01 +0: test/bbb_project_bbb_test.dart: A group of tests First Test 00:01 +1: test/bbb_project_bbb_test.dart: A group of tests First Test 00:01 +1: test/bbb_project_bbb_test.dart: Tesing bbbAdd from package:bbb_project_bbb 00:01 +2: test/bbb_project_bbb_test.dart: Tesing bbbAdd from package:bbb_project_bbb 00:01 +2: All tests passed! bbb_project_bbb: SUCCESS ------------------------------------------------------------------------------------------------------------------------ $ melos exec └> dart test test └> SUCCESS melos run test:dart └> melos exec -c 1 --fail-fast -- "dart test test" └> SUCCESS melos run test:flutter └> melos exec -c 1 --fail-fast -- "flutter test test" └> RUNNING $ melos exec └> flutter test test └> RUNNING (in 1 packages) ------------------------------------------------------------------------------------------------------------------------ccc_project_ccc_example: 00:00 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:01 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:02 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:03 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:04 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:05 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:06 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:07 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:08 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:09 +0: loading /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_te00:09 +0: Tesing cccAdd from package:ccc_project_ccc_example 00:09 +0 -1: Tesing cccAdd from package:ccc_project_ccc_example [E] Invalid argument(s): Failed to load dynamic library 'libaaa_project_aaa.so': libaaa_project_aaa.so: cannot open shared object file: No such file or directory dart:ffi new DynamicLibrary.open package:ccc_project_ccc/src/ffi/io.dart 12:27 createLibraryImpl package:ccc_project_ccc/src/ffi.dart 8:19 createLib package:ccc_project_ccc/ccc_project_ccc.dart 11:24 cccAdd test/ccc_project_ccc_example_test.dart 6:18 main. To run this test again: /home/atom/flutter/bin/cache/dart-sdk/bin/dart test /home/atom/flutter_dart_rust_template/packages/ccc_project_ccc/example/test/ccc_project_ccc_example_test.dart -p vm --plain-name 'Tesing cccAdd from package:ccc_project_ccc_example' 00:09 +0 -1: Some tests failed. ------------------------------------------------------------------------------------------------------------------------ $ melos exec └> flutter test test └> FAILED (in 1 packages) └> ccc_project_ccc_example (with exit code 1) melos run test:flutter └> melos exec -c 1 --fail-fast -- "flutter test test" └> FAILED ScriptException: The script test:flutter failed to execute melos run test └> melos run test:dart --no-select && melos run test:flutter --no-select └> FAILED ```

Does this mean that it is fundamentally impossible to write passing unit tests which rely on the Rust native library, even if the library is bundled with a dependency of the project being tested?

That is to say, the native functionality will be available to the project when built and in integration tests, but not in unit tests?

GregoryConrad commented 1 year ago

Is this true even for packages depending on ccc_project_ccc? I removed the unit tests in ccc_project_ccc, leaving the unit tests in ccc_project_ccc_example. These fail too:

Correct, any unit tests depending on ccc will fail unless they use the binary or mock the implementation.

Does this mean that it is fundamentally impossible to write passing unit tests which rely on the Rust native library, even if the library is bundled with a dependency of the project being tested?

Not necessarily impossible; you have a few options:

  1. Users of ccc can use the binary for their platform to run their unit tests (although I wouldn't necessarily recommend this approach unless the user of ccc is another package in your mono repo).
  2. ccc can provide a stub implementation for testing that does not actually call/use FRB, but I would suggest only allowing this internally in the mono repo, and it could be risky.
  3. Users of ccc mock the ccc functions. IMHO, any unit testing relying on ccc should be doing this anyways, as they are unit tests. Any functionality actually interacting with the outside environment should be an integration test anyways.

All that being said, integration tests should all pass as expected, based on what I see in your repo. I have a feeling you are really looking for integration tests here, not unit tests.

CicadaCinema commented 1 year ago

Thank you for clarifying! I've added an integration test and it does pass, as expected!