Intermittent bindgen failures

[SecondFlight]

I have a case where bindgen succeeds but it somtimes doesn't generate RawTimeSignatureChange in ffigen_binding.dart. It seems random as to whether it will include it or leave it out.

Here's the relevant files in my repo: https://github.com/SecondFlight/anthem/blob/bindgen-race/src/model/pattern.rs https://github.com/SecondFlight/anthem/blob/bindgen-race/src/model/time_signature.rs

This issue first started happening after I added the time_signature_changes and default_time_signature fields to Pattern.

I tried recreating this issue in examples/dart/clock, but to no avail. I was trying to generically recreate the structure in the files linked above:

pub struct Store {
  // ...
  a: Vec<A>,
  b: B,
}

A {
  b: B
}

B {}

This did not cause the issue to occur, so I'm not sure what's happening.

[SecondFlight]

I'm still not sure what's happening, but I've started seeing another odd issue. Now I sometimes get the error below when I'm compiling my Rust code. It only happens sometimes:

PS C:\Users\Joshua\Documents\Code\anthem> cargo run rid_build   
   Compiling anthem v0.1.0 (C:\Users\Joshua\Documents\Code\anthem)
error: symbol `__include_dart_for_vec_u64` is already defined
  --> src\model\store.rs:33:1
   |
33 | #[rid::store]
   | ^^^^^^^^^^^^^
   |
   = note: this error originates in the attribute macro `rid::store` (in Nightly builds, run with -Z macro-backtrace for more info)

error: could not compile `anthem` due to previous error

I've attached a video of what it looks like to run rid_build right now. This time I got each error only once, but it's not uncommon to see both appear multiple times.

https://user-images.githubusercontent.com/6700184/139569129-ae6d9638-f223-4f86-ae96-cdf5d82ec844.mp4

[thlorenz]

Thanks for reporting this. Looks like a race condition of some sort since it only happens sometimes. Basically if you have two cases of a Vec<64> two wrapper methods for it are exported via __include_dart_for_vec_u64 twice which is wrong (should be only one).

Basically rid keeps state internally to see which ones it already rendered but if somehow macro expansion runs in parallel (which I don't think) or if the check if we rendered this happens at the wrong places and still results in two wrapper methods (more likely) then it could cause that issue.

I'm gonna look through the code to see how that could be fixed when I get some more bandwidth. I just switched jobs and have lots on my plate ATM but am sure within a few weeks this should calm down.

Thanks for your patience.

[SecondFlight]

All good! I understand how that goes, no rush :)

I will keep posting as I see new symptoms. I've reproduced this on three different Windows machines and I should have an opportunity to try it on a macOS machine soon.

[SecondFlight]

I can confirm that both errors occur on macOS:

• Sometimes, rid_build fails with "symbol ... is already defined"
• Sometimes, rid_build succeeds but doesn't output part of ffigen_binding.dart

[Roms1383]

Isn't it that the code generator sometimes run too early, while other scripts might still be running ? Maybe sometimes that could be mitigated by looking for file lock ?

Exactly like when running cargo check sometimes the console displays waiting for file lock on build repository if for example your IDE is re-compiling at the same time:

[Roms1383]

I've noticed a redundant failure this early morning because I'm used to chain bash commands (e.g. with Makefile): almost everytime I run make bindgen-m1 && make macos-m1 && make run-macos after modifying code in Rust, on first run it fails at bindgen-m1 but as soon as I re-run it (not modifying anything in-between) then it compiles and runs just fine. There might be something to investigate further.

[SecondFlight]

I don't think that's the issue. I am running the build script directly via cargo run rid_build and waiting for it to complete. Then, I am starting a debug session in VS Code, which I expect is running flutter run -d windows under the hood. Based on this, I believe the issue is in Rid bindgen.

[thlorenz]

@SecondFlight in order to fix this could you please clarify if this happens on Windows only of other OSes as well? Also if you can isolate this at all into a small example that I could run on my machine that'd be great in order to repro and hopefully fix.

[SecondFlight]

Hi @thlorenz, I have reproduced this on Windows and x86 macOS. I was able to test three Windows environments and one macOS environment. I have not tried any other platforms.

I tried to create a minimal reproducible test case in the clock example (see my initial investigation above), but I couldn't reproduce the issue. The issue seems to require either a sufficiently complex setup or a specific structure, I'm not sure which. The repository I linked reproduces the issue reliably on all systems I tested.