FRB v2 Support external rust directory

patmuk commented 8 months ago

Inspired by crux I want to use flutter as a UI-shell: The application core, it’s state, etc, should be in rust, where different shells are sending (mostly user triggered) events to the application core in rust, and receive effects to react to (typically what to render as a result of processing the event).
I made a demo project to implement this setup. Next to a flutter shell you can find a cli shell.
I might be over-engineering this, but I believe that the UI-shell should just be an add-on, so one can support multiple and/or switch in the future.

But I struggle to set this up properly. I believe that this is possible (Worked in FRB pre v1.80.0), but I don’t know if FRB v2 can be configured to do so - and if my approach is correct. This is what I do:

Approach 1:

Have the rust directory as a sibling of the flutter directory, with flutter-codegen.yaml configuring to use that.

My app/
  app_core <- the rust dir
  shell_flutter <- result of flutter_rust_bridge_codegen create, minus /rust
  other shells

currently my library is not found in flutter run, when I moved the folder and adapted the codegen config yaml. I should probably try pointing to the new location in RustLib.init(). I will get into more details on the issues along the way in follow-up comments.

Approach 2:

I was just recently thinking that I could leave the internal rust folder, but import the app_core as a dependency, like:

My App/
  app_core <- the rust dir with the main app logic
  shell_flutter <- result of flutter_rust_bridge_codegen create
     rust <- including FBRs rust crate, which depends on app_core
  other shells

In this solution I would not fall into a trap, where app_core uses some FBR sepecific implementation detail and thus have a too strong coupling. But I don’t want to duplicate the interface code either (which is a handful of functions (process_event, …) and a few structures (Enum Event, …).

currently I get a dependency issue in cargo build, some feature flag of anyhow is not compatible with stable.

Which sounds like the better, more future proof, approach? Keeping a “bridge” rust crate inside the flutter dir, depending on the app_core (and how to generate the needed code without repeating its implementation) or to move it out (making it only one api.rs file in app_core)?

Is there something hardcoded, or a principle problem with either approach?

patmuk commented 8 months ago

Many thanks for reading that long text!

fzyzcjy commented 8 months ago

I don’t know if FRB v2 can be configured to do so - and if my approach is correct.

I also believe this is implementable! There is nothing special in FRB to stop you from doing that IMHO.

currently my library is not found in flutter run

https://cjycode.com/flutter_rust_bridge/guides/how-to/load-library

some feature flag of anyhow is not compatible with stable

Curious what is that? I use anyhow in production with stable and it worked well.

Which sounds like the better, more future proof, approach?

Not having much time to think in depth, but FRB should support all two methods above.

patmuk commented 8 months ago

Thanks! I will do that as a new example in your repository (PR will follow) first - this I can exclude any dependencies I am introducing. I assume that this might be an interesting alternative for others as well (having the app-logic in rust outside of the flutter/dart directory - I will leave the crux-approach to my project).

I haven't decided on the approach - will see if I hit a wall going for one and then switch to the other.

The dependency error I am getting is:

patmuk@SanPats-mini ~/c/o/s/e/f/s/rust (internal_lib_depending_on_app_core)> cargo build
warning: virtual workspace defaulting to `resolver = "1"` despite one or more workspace members being on edition 2021 which implies `resolver = "2"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
note: for more details see https://doc.rust-lang.org/cargo/reference/resolver.html#resolver-versions
   Compiling memchr v2.6.4
   Compiling libc v0.2.151
`
(...)
`
   Compiling flutter_rust_bridge v2.0.0-dev.2
   Compiling chrono v0.4.31
   Compiling oslog v0.2.0
   Compiling lock_api v0.4.11
   Compiling once_cell v1.19.0
   Compiling atomic v0.5.3
   Compiling log v0.4.20
   Compiling hashbrown v0.14.3
   Compiling parking_lot v0.12.1
   Compiling anyhow v1.0.75
error[E0554]: `#![feature]` may not be used on the stable release channel
   --> /Users/patmuk/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/lib.rs:214:32
    |
214 | #![cfg_attr(backtrace, feature(error_generic_member_access))]
    |                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^

   Compiling dashmap v5.5.3
For more information about this error, try `rustc --explain E0554`.
error: could not compile `anyhow` (lib) due to previous error
warning: build failed, waiting for other jobs to finish...`

... but unless you have an add-hoc idea, don't worry, I will minimize the setup first.

fzyzcjy commented 8 months ago

(I will come back to this a little bit later, probably within hours)

fzyzcjy commented 8 months ago

Feel free to PR!

As for the error, try to cargo clean and build again.

patmuk commented 8 months ago

@fzyzcjy Thanks for the feedback! I came back to give it another try (my code is in this branch ).

I am following the setup where the complete rust code is in a folder next to (and not inside) the flutter folder. I did not implemented an own custom library loader, as that doesn't look so easy from the first glance. It would be great if that is not needed and rust's build folder is found by specifying 'rust_root'. Or a new option 'rust_target_dir'?

However, I think my code fails before that - can it be that the rust directories are somewhere hardcoded?

I can generate & compile, all ends up in the correct folders. But when flutter run Xcode has an error compiling for the Simulator:

just run
cargo lipo
[INFO  cargo_lipo::meta] Will build universal library for ["app_core"]
[INFO  cargo_lipo::lipo] Building "app_core" for "aarch64-apple-ios"

(...)

    Finished dev [unoptimized + debuginfo] target(s) in 0.35s
[INFO  cargo_lipo::lipo] Building "app_core" for "x86_64-apple-ios"

(...)

    Finished dev [unoptimized + debuginfo] target(s) in 0.16s
[INFO  cargo_lipo::lipo] Creating universal library for app_core
cp target/universal/debug/libapp_core.a shell_flutter/build/ios/Debug-iphonesimulator/rust_builder/librust_lib.a

Here is the interesting part:

cd shell_flutter && flutter run
Launching lib/main.dart on iPhone 15 Pro in debug mode...
Running Xcode build...                                                  
Xcode build done.                                            4.2s
Failed to build iOS app
Error output from Xcode build:
↳
    ** BUILD FAILED **

Xcode's output:
↳
    Writing result bundle at path:
        /var/folders/65/4zdp9whj28j122crpwzk7d9m0000gs/T/flutter_tools.q
        Hs8zY/flutter_ios_build_temp_dirGBnGBW/temporary_xcresult_bundle

    TARGET_DEVICE_PLATFORM_NAME=iphonesimulator

(...)

    SEVERE:
    =================================================================
    ===============
    SEVERE: Cargokit BuildTool failed with error:
    SEVERE:
    -----------------------------------------------------------------
    ---------------
    SEVERE: PathNotFoundException: Cannot open file, path =
    '/Users/patmuk/code/own/sherry/examples/flutter-rust-bridge_crux_
    style/shell_flutter/ios/Pods/../.symlinks/plugins/rust_builder/io
    s/../../rust/Cargo.toml' (OS Error: No such file or directory,
    errno = 2)

I removed the nested rust folder. And set:

rust_input: ../app_core/src/api.rs
rust_root: ../app_core
rust_output: ../app_core/src/bridge/generated/mod.rs

Is it hardcoded somewhere? I could only see that cargokit takes it from env, but I did not find the place where cargokit is called.

Here is the rest of the stack trace:

    SEVERE:
    -----------------------------------------------------------------
    ---------------
    SEVERE: #0      _File.throwIfError (dart:io/file_impl.dart:675:7)
    SEVERE: #1      _File.openSync (dart:io/file_impl.dart:490:5)
    SEVERE: #2      _File.readAsBytesSync
    (dart:io/file_impl.dart:574:18)
    SEVERE: #3      _File.readAsStringSync
    (dart:io/file_impl.dart:624:18)
    SEVERE: #4      CrateInfo.load
    (package:build_tool/src/cargo.dart:45:35)
    SEVERE: #5      BuildEnvironment.fromEnvironment
    (package:build_tool/src/builder.dart:92:33)
    SEVERE: #6      BuildPod.build
    (package:build_tool/src/build_pod.dart:31:42)
    SEVERE: #7      BuildPodCommand.runBuildCommand
    (package:build_tool/src/build_tool.dart:51:17)
    SEVERE: #8      BuildCommand.run
    (package:build_tool/src/build_tool.dart:37:11)
    SEVERE: #9      CommandRunner.runCommand
    (package:args/command_runner.dart:212:27)
    SEVERE: #10     CommandRunner.run.<anonymous closure>
    (package:args/command_runner.dart:122:25)
    SEVERE: #11     new Future.sync (dart:async/future.dart:305:31)
    SEVERE: #12     CommandRunner.run
    (package:args/command_runner.dart:122:14)
    SEVERE: #13     runMain
    (package:build_tool/src/build_tool.dart:251:18)
    SEVERE: #14     runMain (package:build_tool/build_tool.dart:7:21)
    SEVERE: #15     main
    (file:///Users/patmuk/Library/Developer/Xcode/DerivedData/Runner-
    blmuhmhlinujwtcnoyumofmstbip/Build/Intermediates.noindex/Pods.bui
    ld/Debug-iphonesimulator/rust_builder.build/build_tool/bin/build_
    tool_runner.dart:3:14)
    SEVERE: #16     _delayEntrypointInvocation.<anonymous closure>
    (dart:isolate-patch/isolate_patch.dart:295:33)
    SEVERE: #17     _RawReceivePort._handleMessage
    (dart:isolate-patch/isolate_patch.dart:184:12)
    SEVERE:
    -----------------------------------------------------------------
    ---------------

The lib name "rust_lib" looks hardcoded as well - as I removed the manifest with the rust folder ...

    SEVERE: BuildTool arguments: [build-pod, ../../rust, rust_lib]
    SEVERE:
    =================================================================
    ===============
    Command PhaseScriptExecution failed with a nonzero exit code
    /Users/patmuk/code/own/sherry/examples/flutter-rust-bridge_crux_s
    tyle/shell_flutter/ios/Pods/Pods.xcodeproj: warning: The iOS
    Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set
    to 11.0, but the range of supported deployment target versions is
    12.0 to 17.2.99. (in target 'rust_builder' from project 'Pods')

(...)

Could not build the application for the simulator.
Error launching application on iPhone 15 Pro.
error: Recipe `run` failed on line 37 with exit code 1

Can you give me a hint where to look at?

fzyzcjy commented 8 months ago

(I will check this a bit later probably in a few hours)

fzyzcjy commented 8 months ago

Is it hardcoded somewhere?

FRB's template is setup like this: Follow https://matejknopp.com/post/flutter_plugin_in_rust_with_no_prebuilt_binaries/ and create a Flutter package named rust_builder, and it points to the rust folder for Rust source. Thus, I guess you can firstly read that article, then it is a bit easier to follow the logic of cargokit. e.g. it is hardcoded in android build.gradle, ios podfile, etc.

Btw, if I understand correctly, the proposal is to move the ./rust into another parallel folder. Then I wonder whether it is needed to move it, or directly keep it in the original place

patmuk commented 8 months ago

Thanks! I will read and try that.

Yes, I want to use flutter as a UI only, managing even the state in rust. My structure looks like:

app_core (in rust)
shell_flutter (flutter UI)
Shell_cli (another UI, here a cli in rust)

So, yes, I want to move shell_flutter/rust to app_core. An alternative could be to keep it there and have app_core as a dependency - that I will do if having it as a parallel folder is to complicated. But in this approach I am afraid that I would need to duplicate the api, as this should be used by shell_cli as well (though if frb generates wires for imported public functions and structs this wouldn't be required ...)

fzyzcjy commented 8 months ago

I see - then it seems reasonable to not use the default ./rust folder!

patmuk commented 8 months ago

I got it working!

@fzyzcjy strangely I had to add dependencies to freezed. That being missing should be a problem for anyone?

For anyone interested in the same setup, here is what I changed (in brief - @fzyzcjy if you like I could write that up for the documentation - just point me to the right place.):

delete the folder 'rust' - as you will replace it with your own folder (app_core in my case)

adapt the build configuration, replacing the relative path to your rust project (from ../../rust to, in my case, ../../../app_core (mind the extra ../!)):

diff --git a/shell_flutter/rust_builder/android/build.gradle b/shell_flutter/rust_builder/android/build.gradle
index a8517ac..83cbdd9 100644
--- a/shell_flutter/rust_builder/android/build.gradle
+++ b/shell_flutter/rust_builder/android/build.gradle
@@ -51,6 +51,6 @@ android {

apply from: "../cargokit/gradle/plugin.gradle"
cargokit {
-    manifestDir = "../../rust"
-    libname = "rust_lib"
+    manifestDir = "../../../app_core"
+    libname = "app_core"
}
diff --git a/shell_flutter/rust_builder/ios/rust_builder.podspec b/shell_flutter/rust_builder/ios/rust_builder.podspec
index 24a8ffd..11d72d4 100644
--- a/shell_flutter/rust_builder/ios/rust_builder.podspec
+++ b/shell_flutter/rust_builder/ios/rust_builder.podspec
@@ -29,17 +29,17 @@ A new Flutter FFI plugin project.
s.script_phase = {
 :name => 'Build Rust library',
 # First argument is relative path to the `rust` folder, second is name of rust library
-    :script => 'sh "$PODS_TARGET_SRCROOT/../cargokit/build_pod.sh" ../../rust rust_lib',
+    :script => 'sh "$PODS_TARGET_SRCROOT/../cargokit/build_pod.sh" ../../../app_core app_core',
 :execution_position => :before_compile,
 :input_files => ['${BUILT_PRODUCTS_DIR}/cargokit_phony'],
 # Let XCode know that the static library referenced in -force_load below is
 # created by this build step.

run flutter run

fzyzcjy commented 8 months ago

strangely I had to add dependencies to freezed. That being missing should be a problem for anyone?

when you need freezed (i.e. using nontrivial rust enums), it is needed. Feel free to create issue and/or PR to, e.g. (a) error when users do not add freezed as dependency, or (b) directly help them add a dependency. If you need, ping me for more hints about implementing this!

if you like I could write that up for the documentation - just point me to the right place

feel free to create a separate issue to track this. I may need to think a bit which exact location to add it...

patmuk commented 8 months ago

strangely I had to add dependencies to freezed. That being missing should be a problem for anyone?

when you need freezed (i.e. using nontrivial rust enums), it is needed. Feel free to create issue and/or PR to, e.g. (a) error when users do not add freezed as dependency, or (b) directly help them add a dependency. If you need, ping me for more hints about implementing this!

Ah, that makes sense :) I will create an issue for that. Though it is quite obvious that freezes has to be added, the error message states that quite clearly.

if you like I could write that up for the documentation - just point me to the right place

feel free to create a separate issue to track this. I may need to think a bit which exact location to add it...

Ok, I will make a PR writing it up - you can later change that :)

patmuk commented 8 months ago

~~@fzyzcjy for the documentation, are the sources in main or in gh-pages? I see only html files, though I assume the sources are in md?~~

Found it!

patmuk commented 8 months ago

Documented in #1547

patmuk commented 7 months ago

automatic support will be implemented in #1567

patmuk commented 6 months ago

Closing this issue, as support has been implemented (in #1567). Tested and can confirm it works.

A "how to" can be found in https://cjycode.com/flutter_rust_bridge/guides/custom/codegen (basically use the option --rust-crate-dir ../app_core when having the folder structure outlined in my first comment).

Many thanks @fzyzcjy! (And please excuse my silence - I was in holidays.

fzyzcjy commented 6 months ago

You are welcome and happy to see it works!

github-actions[bot] commented 6 months ago

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new issue.

fzyzcjy / flutter_rust_bridge

FRB v2 Support external rust directory #1459

Approach 1:

Approach 2: