search

fzyzcjy / flutter_rust_bridge

Flutter/Dart <-> Rust binding generator, feature-rich, but seamless and simple.
https://fzyzcjy.github.io/flutter_rust_bridge/
MIT License
4.24k stars 291 forks source link

Error: function=Ident { sym: iter, span: bytes(6611009..6611013) } - 1: Unknown ident: Iterator #1990

Closed Larpoux closed 4 months ago

Larpoux commented 5 months ago

This issue is more on less related to #1903.

I really need to give an access to flutter users to the library web-audio-api-rs. This library is huge (several hundreds of very sophisticated modules).

Actually I don't consider writing a wrapper around this library because of the hard work it will be and because I am very new with Rust and don't master it. I began to hack the source code of the library so that it will be OK for frb and will get (one day soon I hope) at least a partially working dart interface.

I have several requirements that I will enter in several different issues.

My first requirement is that I get for a reason or an another the following error during generate : 

[15.7s] Parse  
  └── [0.5s] Run cargo expand  
  └── [0.3s] Parse source graph                                                                                                                                                                    Error: function=Ident { sym: iter, span: bytes(6611009..6611013) }

Caused by:
    0: when trying to parse DartFn
    1: Unknown ident: Iterator
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s

The big problem is that I have no clue of the reason of the crash. Neither the rust file nor the line number that triggered the crash. And I do not get any generated dart files.

Here are the Verbose logs

fzyzcjy commented 5 months ago

The big problem is that I have no clue of the reason of the crash.

Looks like the crate uses some grammar that frb does not understand (yet). A quick guess would be impl Fn() -> ... with unsupported types.

Actually I don't consider writing a wrapper around this library because of the hard work it will be and because I am very new with Rust and don't master it. I began to hack the source code of the library so that it will be OK for frb and will get (one day soon I hope) at least a partially working dart interface.

I am not very sure whether this will work... Indeed writing a wrapper may be easier. Just use the "newtype pattern" everywhere and done!

For example, suppose 3rd party has

pub struct A { ... }
impl A {
  pub fn f(&self) -> String {...}
}

Then the standard so-called "new type pattern" is nothing but a wrapper:

// in your crate
pub struct A(audio_crate::A);

impl A {
  pub fn f(&self) -> String {
    self.0.f() // just delegate to function on `self.0`, the real 3rd party crate
  }
}
Larpoux commented 5 months ago

Thank you @fzyzcjy for your advice. You are probably right because you know perfectly frb and because you master Rust. However, during several days, I will try to continue working on modifying the lib sources so that I will get a first draw of something that dart will accept. If I do not success I will fallback to write a wrapper. But I am very anxious about the amount of work to do that.

Again, thank you for this amazing tool. I will enter several different issues. Just close them if you think that my requirements are stupid.

fzyzcjy commented 5 months ago

What if we partially handle #1903? Like:

Larpoux commented 5 months ago

Yes @fzyzcjy : I would be really happy with something like that 👍

fzyzcjy commented 5 months ago

Then that's great!

To be honest, that is not a tiny workload to implement indeed, but I will try to squeeze time for that (can it be within several day or a week?).

Larpoux commented 5 months ago

No problem with delay. We have no clients nor boss. We are free. We just code because this is the best thing we can do for our life. Free software is something incredible.

fzyzcjy commented 5 months ago

Totally agree!

fzyzcjy commented 5 months ago

Btw I realize my English maybe wrong above:

can it be within several day or a week?

I meant, I wonder whether the time I find in the future will be within several days or a week. But I suddenly realize that, maybe this has another interpretation that, I ask you to confirm a deadline for me. No worries, I do not mean that!

Maybe I should say

maybe it is within several day or a week?

fzyzcjy commented 4 months ago

Btw I have started: https://github.com/fzyzcjy/flutter_rust_bridge/pull/2000

Larpoux commented 4 months ago

@fzyzcjy : I am so Happy that you are working on my requests 👍 . Actually, I am now able to run "flutter_rust_bridge_codegen" on a modified fork of the lib, and I am able to generate a dart interface to it.

But I am more and more thinking that you was right : the good way for me is to write a dart wrapper around the lib.

    • I had to modify the lib so that "flutter_rust_bridge_codegen" can run until end without stopping on error
    • I really would like to interface the real crate lib and not a modified version because of maintenance reasons
    • Many PUB struct and many PUB functions are PUB because they are needed internally by the lib, but are not to be offered to the lib clients like me.
    • I have no ideas how to use my generated dart interface with the official rust crate : the way how dart is calling a rust lib is actually a bit of magic for me
    • Ultimately I would like a dart interface being exactly the interface that google offers for Web users, and not a derivative interface offers by my Rust lib and generated by your tool

The big problems are :

Thank you @fzyzcjy for your work. I appreciate.

fzyzcjy commented 4 months ago

You are welcome!

P.S. I plan to firstly create a minimal version, and we can check what is most critically missing, and work on that, and check again, etc.

fzyzcjy commented 4 months ago

P.S. In case you are interested, demo may be in https://github.com/fzyzcjy/demo_web_audio_api_flutter (just for me to play with flutter_rust_bridge when working on frb PRs)

Larpoux commented 4 months ago

Where is the repo of the dev branch ? I am using the master branch of flutter_rust_bridge which seems to be 'version = "2.0.0-dev.37"' but I have still the error message : 

Error: Do not use `lib.rs` as a Rust input. Please put code to be generated in something like `api.rs`.

I tried just to be helpful for you, but it seems that you are testing with 'web-audio-api` 👍 , so you do not need my own feedback now. I suggest that I will try when you will be ready. I am not in a hurry : I have many, many things to do on the dart side.

fzyzcjy commented 4 months ago

I am on feat/12109 of flutter_rust_bridge currently, but it is just one of the PRs - when this is done I will move on the next branch corresponding to next PR, etc.

Yes, I guess I will firstly work on frb, and when it supports a minimal set of things related to third party crates will ping you!

Error: Do not use lib.rs as a Rust input. Please put code to be generated in something like api.rs.

Btw that does not look like latest master branch (maybe you did not git pull, or somehow is using published version?) Anyway no problems, I will release new versions later!

fzyzcjy commented 4 months ago

https://github.com/fzyzcjy/demo_web_audio_api_flutter now compiles. This very initial version only generates a small portion of the APIs currently - I am still working on frb to make it auto generate more.

Larpoux commented 4 months ago

Hi @fzyzcjy , I tried the current frb and it seems very encouraging. I like that it is now possible to declare several rust_input: and that the generations does not stop during its process. And that it is not mandatory to download the lib sources.

I am impressed that you know Rust and Dart so well. Congratulations.

Larpoux commented 4 months ago

Sorry, encouraging and not uncourageous

Larpoux commented 4 months ago

For your information : the dart compiler show : 

Error (Xcode): ../lib/src/rust/frb_generated.dart:77:42: Error: Type 'DummyStruct' not found.
fzyzcjy commented 4 months ago

For your information : the dart compiler show :

No worries, the PRs are not merged yet and even master branch does not work currently!

fzyzcjy commented 4 months ago

Update:

image

Larpoux commented 4 months ago

There are two things missing in the documentation :

  1. There is nothing about using frb in a flutter plugin. I do not remember what I did, but I only remember that it was not straightforward.
  2. There is a topic about debugging the dart code, but nothing about debugging the rust code, which is really what we need. I am now able to debug the rust code (breakpoints, show variables value, ....) under Xcode. This is something very important. I have not been able to debug on Android, but I will try again later : I focus actually on iOS

When I will finish my project, I will try to remember what I did, and perhaps add two topics in the doc.

PS: (No problem with the code size. Everything OK now, for me).

fzyzcjy commented 4 months ago

Totally agree and looking forward to the docs!

Btw, for the 1st point, the short story may be to follow https://github.com/irondash/cargokit's tutorial, and add flutter_rust_bridge.yaml.

PS. current progress: Working on refactoring HIR (high-level intermediate representation, a module in frb parser) in order to enable more complex parsing to support features to parse web audio.

Larpoux commented 4 months ago

It is now possible to specify several libraries during the dart interface generation. This is really great. Do you think possible to add a comment in front of the generated dart files ?

/// {@category name_of_the_api}
library my_library;

This way it will be possible to separate the different documentation files of each API. see dartdoc documtentation

Larpoux commented 4 months ago

And maybe a subcategory when cascading in mod.rs : 

/// {@category name_of_the_api}
/// {@subCategory name_of_mod_rs}
library my_library;

(I am not very sure of this request ....)

fzyzcjy commented 4 months ago

Looks interesting, and feel free to create a new issue for more discussions!

Btw I plan to make a release soon which provides a web-audio-api binding that can compile and covers (hopefully) many APIs and looking forward to your trying when released :)

Larpoux commented 4 months ago

Actually frb generates dart files in the src subfolder. This means that those modules are supposed to be private. This is OK for the frb_generated.dart. But my needs are to offer the api/ dart files as public interface. This is no problem to automaticate this with a perl script or something like that, and change the referenced imported modules in frb_generated.dart to point to ../public/

I don't really know if this need is universal. If yes, probably good if frb does that itself. If no, I will do those scripts. No problem.

fzyzcjy commented 4 months ago

IMHO a common way in Dart seems to be write down a file, say, lib/flutter_sound.dart, which has content like:

export 'src/api/a.dart';
export 'src/api/b.dart';
export 'src/api/c.dart';
Larpoux commented 4 months ago

files in src/ are not processed by dart doc

fzyzcjy commented 4 months ago

Btw just curious, may I know the link to your current repo about the experiments? Then I may have a look and know a bit about how the generated code is to be called.

fzyzcjy commented 4 months ago

files in src/ are not processed by dart doc

Hmm, but export seems to work? At least for flutter_rust_bridge's package that seems to be true

More specifically, files in src seems to be ignored, but file at lib/something.dart (not in lib/src) which export something in src does work

Larpoux commented 4 months ago

I have in my main module : 

export 'package:tau/src/rust/api/simple.dart';
export 'package:tau/src/rust/api/toto.dart';

but toto.dart and simple.dart are not processed by dart doc

fzyzcjy commented 4 months ago

That's weird!

Looking at https://pub.dev/documentation/flutter_rust_bridge/2.0.0-dev.37/flutter_rust_bridge/flutter_rust_bridge-library.html, it seems that it understands things inside src folder and exported.

Could you please provide a link to the repo such that I can have a brief look?

Larpoux commented 4 months ago

this is my repo

cd tau
rm -r doc/api/*
dart doc .
dart pub global run dhttpd --path doc/api
Larpoux commented 4 months ago

I am very late on my project. I spend all my time on Flutter Sound maintenance. More, when I was young I was fast, too fast. Now that I am old I am slow, too slow.

I am very happy if I can avoid to write all the rust interface, thanks to your work. But I am not sure if I will have to write a dart interface to rust : Google offers a dart interface to the JavaScript Web Audio API : this file

I am afraid that I will have to write something equivalent, specially if I want to be 100% compatible with Web app. Actually I ported the web-audio-api-rs simple AudioPanner example to my project. This is just a copy and past of the rust code inside simple.rs. It is called by my example/lib/RustEx/stereo_panner.dart It works perfectly under iOS.

I want now to write the same functions in dart, calling web-audio-api.rs . This will be simple if I don't have to write a dart interface, like Google does.

Larpoux commented 4 months ago

BTW : Dart convention is that words inside squared braquet in comments are name of another classname. Dart doc uses this convention to put links to the class documentation. This convention is not same in Rust. When I run dart doc I have ton of warnings that these are not valid class names. I am doubtful that frb can do something for that. It is not frb job to analyse the comments and do something when it's not a classname. I think I will have to live with those warnings.

Larpoux commented 4 months ago

I am specially anxious about memory leaks. People create contexts and nodes under JavaScript and do not bother to free anything. But they will have to dispose those rust objects. I wonder if I can relay on the dart finalize mechanism so that users will not have this responsibility.

fzyzcjy commented 4 months ago

I am very late on my project. I spend all my time on Flutter Sound maintenance.

Hmm I am a bit confused, IIRC you mentioned web-audio-rs will be used under the hood in future flutter_sound, thus working on this seems the same thing as flutter sound maintenance? (Forgive me if I understand wrongly - I gets a bit confused between tau and flutter_sound)

More, when I was young I was fast, too fast. Now that I am old I am slow, too slow.

No worries, it's OK!

I am very happy if I can avoid to write all the rust interface, thanks to your work. But I am not sure if I will have to write a dart interface to rust : Google offers a dart interface to the JavaScript Web Audio API : this file I want now to write the same functions in dart

Yes, that's what I hope to do - enhance flutter_rust_bridge, such that you can write something very similar to the Rust counterpart, without manually doing very little extra work!

Currently it is something like below (I port from another demo, but things are similar IMHO):

https://github.com/fzyzcjy/flutter_rust_bridge/blob/76cdb39ad38c2effe2010c463900cf70c26784e7/frb_example/integrate_third_party/integration_test/simple_test.dart#L21

Still missing a bit (the commented out lines), but I am working on it and those lines will work as well soon.

This convention is not same in Rust. I am doubtful that frb can do something for that. It is not frb job to analyse the comments and do something when it's not a classname.

I can try to do something for that. Maybe very naively parse the comments strings, replace all such things. For example, [Apple] orange --> \Apple` orange`.

I am specially anxious about memory leaks. I wonder if I can relay on the dart finalize mechanism so that users will not have this responsibility.

flutter_rust_bridge already uses Dart finalize mechanism! When a Dart object is not used anymore and GCed (garbage collected), the corresponding Rust object will be auto released as well :)

Larpoux commented 4 months ago

Hi @fzyzcjy . I am not sure how to test your dev. When I do this : 

git clone https://github.com/fzyzcjy/flutter_rust_bridge.git
cd flutter_rust_bridge.git
git pull https://github.com/fzyzcjy/flutter_rust_bridge refs/pull/2045/head
cd frb_codegen
cargo build

I get this error : 

   Compiling flutter_rust_bridge_codegen v2.0.0-dev.37 (/Volumes/mac-J/larpoux/projmac/flutter_rust_bridge/frb_codegen)
error[E0425]: cannot find value `TODO` in this scope
  --> frb_codegen/src/library/codegen/parser/hir/flat/transformer/generate_trait_impl_enum_transformer.rs:63:30
   |
63 |     let blocking_read_body = TODO;
   |                              ^^^^ not found in this scope

error[E0425]: cannot find value `TODO` in this scope
  --> frb_codegen/src/library/codegen/parser/hir/flat/transformer/generate_trait_impl_enum_transformer.rs:64:31
   |
64 |     let blocking_write_body = TODO;
   |                               ^^^^ not found in this scope

error[E0425]: cannot find value `enum_def` in this scope
  --> frb_codegen/src/library/codegen/parser/hir/flat/transformer/generate_trait_impl_enum_transformer.rs:85:11
   |
85 |         "{enum_def}
   |           ^^^^^^^^ not found in this scope

error[E0425]: cannot find value `TODO` in this scope
  --> frb_codegen/src/library/codegen/parser/hir/flat/transformer/generate_trait_impl_enum_transformer.rs:91:5
   |
91 |     TODO
   |     ^^^^ not found in this scope

error[E0425]: cannot find value `variants` in this scope
   --> frb_codegen/src/library/codegen/parser/hir/flat/transformer/generate_trait_impl_enum_transformer.rs:116:14
    |
116 |             {variants}
    |              ^^^^^^^^ not found in this scope

For more information about this error, try `rustc --explain E0425`.
fzyzcjy commented 4 months ago

I am not sure how to test your dev.

No it is not ready yet! I will release a new version and you can use that later.

Larpoux commented 4 months ago

This convention is not same in Rust. I am doubtful that frb can do something for that. It is not frb job to analyse the comments and do something when it's not a classname.

I can try to do something for that. Maybe very naively parse the comments strings, replace all such things. For example, [Apple] orange --> \Appleorange.

Not good : there are very often [aClassName] inside the rust comments. They are correctly translated with a Web Link to the class documentation. This is very handy. And when Dart Doc does not find the class, it correctly write the thing in highlight. Better to live with the warnings and have the class correctly referenced with a web link.

fzyzcjy commented 4 months ago

Ah I see, then maybe we should keep it untouched!

fzyzcjy commented 4 months ago

@Larpoux Hi, 2.0.0.dev-38 is now released and feel free to have a try!

The doc is in https://cjycode.com/flutter_rust_bridge/guides/third-party (indeed https://cjycode.com/flutter_rust_bridge/guides/third-party/automatic for this specific scenario)

The code is in https://github.com/fzyzcjy/flutter_rust_bridge/tree/master/frb_example/integrate_third_party

A demo usage looks like https://github.com/fzyzcjy/flutter_rust_bridge/blob/dcf73a91b2a19eedcb3d3128d762abd1f3d6a4c6/frb_example/integrate_third_party/integration_test/simple_test.dart#L21

Larpoux commented 4 months ago

@fzyzcjy , thank you so much for your fantastic work. Unfortunately I am not ready to do extensive tests : I began to reorganize all the project so that the lib will be published independently of flutter. I did this fo Flutter Sound and I am glad with that. This way, the lib will be usable by other platforms, like React Native.

I am going to do a very simple test to try Frb. I will let you know the results.

@fzyzcjy , I would like to drink a beer with you. Please give me your private email address so that I will send you 10 euros with PayPal for a lager pint of beer.

Larpoux commented 4 months ago

Mine is Larpoux at gmail.com

Larpoux commented 4 months ago

When I generate the web-audio-api it generates only 8 modules. Do I do something wrong ?

target/debug/flutter_rust_bridge_codegen -V
flutter_rust_bridge_codegen 2.0.0-dev.38
fzyzcjy commented 4 months ago

@Larpoux You are welcome!

Unfortunately I am not ready to do extensive tests : I began to reorganize all the project so that the lib will be published independently of flutter. I did this fo Flutter Sound and I am glad with that. This way, the lib will be usable by other platforms, like React Native. I am going to do a very simple test to try Frb. I will let you know the results.

Take your time, and looking forward to the results!

I would like to drink a beer with you. Please give me your private email address so that I will send you 10 euros with PayPal for a lager pint of beer.

Thank you very much for that! I write open source not for money, so there is no need to give me any money and I will try my best as always to work on it. If you still want to send me a beer, just donate to people who need it and I am equally happy. Btw if you want to discuss with me by email, you can reach me at ch271828n at outlook dot com. (Not sure whether I expressed my mind clearly since my English is not very good)

fzyzcjy commented 4 months ago

When I generate the web-audio-api it generates only 8 modules. Do I do something wrong ?

It should generate something like https://github.com/fzyzcjy/flutter_rust_bridge/tree/master/frb_example/integrate_third_party/lib. Indeed one way to play with it is to directly clone the repo and open that integrate_third_party Flutter project.

I do expect some things are missing, such as complex generics etc. Only had time to check this demo https://github.com/fzyzcjy/flutter_rust_bridge/blob/b7e4a083a87f4b1d4e6a4ce29094d55ac37c1a4e/frb_example/integrate_third_party/integration_test/simple_test.dart#L21, and it does compile.

Looking at https://github.com/fzyzcjy/flutter_rust_bridge/tree/master/frb_example/integrate_third_party/lib/src/rust/third_party/web_audio_api, yes there seems to be only a few files. This seems to be because web_audio_api Rust lib chooses to do a lot of pub use something::*, and then flutter_rust_bridge considers all things inside something to be in the outer mod, instead of the inner mod.

fzyzcjy commented 4 months ago

@Larpoux Btw I replied to the email yesterday, not sure whether it enters your inbox (I saw once in a while some normal email gets to my junk mail box instead of inbox...) thus I resay here

Larpoux commented 4 months ago

Yes @fzyzcjy , I received your mail and I thanks you for it.

Now, we are to the point where some of the Rust modules of web-audio-api-rs are accessible from Dart, thanks to FRB. But very important modules are skipped. Could you tell me if there is any possibility that those modules will be processed by FRB in the foreseeable future ? Or if I have to begin today to write a rust wrapper above those functions, as you advised me to do ? I can do that if necessary, but I don't master Rust (which is a very difficult language for me), and I am afraid that I will just have to do things that FRB does (opaque structures, Reference Counts management, ...).

Larpoux commented 4 months ago

BTW : I would like to help on FRB development, but I have to learn RUST very well before that. And it will not be easy.