search

shekohex / flutterust

Flutter + Rust = :heart:
Apache License 2.0
699 stars 56 forks source link

Have to make clean everytime I change rust code #21

Open tbraun96 opened 3 years ago

tbraun96 commented 3 years ago

Unfortunately, in order to compile the rust program using cargo make, I must first run cargo make clean. This, of course, adds a long compile-time overhead. This is the error message I get when trying to cargo make without cleaning first:

[cargo-make] INFO - cargo make 0.32.12
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: default
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: workspace
[cargo-make][1] INFO - Project: scrap-ffi
[cargo-make][1] INFO - Build File: Makefile.toml
[cargo-make][1] INFO - Task: default
[cargo-make][1] INFO - Profile: development
[cargo-make][1] INFO - Execute Command: "cargo" "fmt"
[cargo-make][1] INFO - Running Task: setup-crate-type
[cargo-make][1] INFO - Running Task: pre-android
[cargo-make][1] INFO - Execute Command: "cargo" "ndk" "--platform" "30" "--target" "aarch64-linux-android" "build"
warning: output filename collision.
The lib target `scrap-ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)` has the same output filename as the lib target `scrap-ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)`.
Colliding filename is: /Users/nologik/satori.net/flutterust/target/aarch64-linux-android/debug/deps/libscrap_ffi.a
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `scrap-ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)` has the same output filename as the lib target `scrap-ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)`.
Colliding filename is: /Users/nologik/satori.net/flutterust/target/aarch64-linux-android/debug/libscrap_ffi.a
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
thread 'main' panicked at 'assertion failed: mtimes.insert(output.clone(), mtime).is_none()', src/tools/cargo/src/cargo/core/compiler/fingerprint.rs:996:13
stack backtrace:
   0:        0x1016ca1d4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hb0278541eabdb65b
   1:        0x1016e9cdc - core::fmt::write::h6eab26c41e1615a6
   2:        0x1016c26b0 - std::io::Write::write_fmt::h53012594e7e61aa3
   3:        0x1016cca70 - std::panicking::default_hook::{{closure}}::h3d38db2d154383b2
   4:        0x1016cc598 - std::panicking::default_hook::hb9d33e9315d46c05
   5:        0x1016cd024 - std::panicking::rust_panic_with_hook::hcf65477e7471a4ca
   6:        0x1016ccbd8 - std::panicking::begin_panic_handler::{{closure}}::h98a6978c37ed787f
   7:        0x1016ca68c - std::sys_common::backtrace::__rust_end_short_backtrace::h989ccc6ff53356f4
   8:        0x1016ccb68 - _rust_begin_unwind
   9:        0x10173d894 - core::panicking::panic_fmt::h72cacefe63e11063
  10:        0x10173d820 - core::panicking::panic::h0416c1ef77139eda
  11:        0x1011d73c4 - cargo::core::compiler::fingerprint::calculate::ha7ebd2e8d5375aa7
  12:        0x1011d1bec - cargo::core::compiler::fingerprint::prepare_target::h9e061c15f1b23525
  13:        0x101326b5c - cargo::core::compiler::compile::h8e1a5b2363291b32
  14:        0x1010adb5c - cargo::core::compiler::context::Context::compile::hf9bb50277194baa7
  15:        0x10128c468 - cargo::ops::cargo_compile::compile_ws::h9ba552f8d250f9be
  16:        0x10128c1e0 - cargo::ops::cargo_compile::compile::h169304b304473a16
  17:        0x101014404 - cargo::commands::build::exec::h99c42455ffd71d1e
  18:        0x10102a340 - cargo::cli::main::h109414b41770b785
  19:        0x10102394c - cargo::main::hb4945485930fa182
  20:        0x10101ce00 - std::sys_common::backtrace::__rust_begin_short_backtrace::h31ccbcc83be8cd7a
  21:        0x10101cfcc - std::rt::lang_start::{{closure}}::h2b1de5896d1ddb85
  22:        0x1016cd46c - std::rt::lang_start_internal::hfcf8d8a4320307b0
  23:        0x1010257a4 - _main
[cargo-make][1] ERROR - Error while executing command, exit code: 101
[cargo-make][1] WARN - Build Failed.
[cargo-make] ERROR - Error while executing command, exit code: 1
[cargo-make] WARN - Build Failed.

Additionally, between every compile, I need to manually change the cargo.toml. Before compile, crate-type must have both cdylib and staticlib in order to compile:

[lib]
name = "scrap_ffi"
crate-type = ["staticlib", "cdylib"]

After running cargo make, it gets changed to:

[lib]
name = "scrap_ffi"
crate-type = ["staticlib", "staticlib"]

If I forget to change the crate-type back to including cdylib, then compiling fails

Note, as long as I complete the workarounds, the program compiles fine. It's just that there's additional compile-time overhead in having to clean the workspace and resetting the cargo.toml manually each time

shekohex commented 3 years ago

that's really weird behavior, what version of cargo make do you use? could you please try with a clean local clone and try running cargo make android and if it worked, try the cargo make ios? I don't have a macOS machine right now, but I used to cargo make android and it works properly (also in the CI).

I see in these error messages a warning about Colliding filename here:

warning: output filename collision. The lib target scrap-ffi in package scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi) has the same output filename as the lib target scrap-ffi in package scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi). Colliding filename is: /Users/nologik/satori.net/flutterust/target/aarch64-linux-android/debug/deps/libscrap_ffi.a The targets should have unique names. Consider changing their names to be unique or compiling them separately.

Could you try with a clean clone and for example try only one target? ios or android?

tbraun96 commented 3 years ago

Hi @shekohex,

You brought up a good point that maybe one of my build dependencies was out of date. I was using cargo-ndk 1.0. I upgraded it to that latest version, and upgraded to the most recent Makefile.toml in this repo. I checked to see if this fixed the problem. I compiled the program using cargo make, then tried cargo make again to see if the same error would occur (remember: error occurs when i don't run cargo make clean before running cargo make). Strangely, the compiling works fine on Android, but, once it gets to compiling the targets for ios via lipo:

[INFO  cargo_lipo::meta] Will build universal library for ["scrap-ffi"]
[INFO  cargo_lipo::lipo] Building "scrap-ffi" for "aarch64-apple-ios"
warning: output filename collision.
The lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)` has the same output filename as the lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)`.
Colliding filename is: /Users/nologik/satori.net/flutterust/target/aarch64-apple-ios/debug/deps/libscrap_ffi-01ed18e308d8ab79.a
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)` has the same output filename as the lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)`.
Colliding filename is: /Users/nologik/satori.net/flutterust/target/aarch64-apple-ios/debug/libscrap_ffi.a
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)` has the same output filename as the lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)`.
Colliding filename is: /Users/nologik/satori.net/flutterust/target/aarch64-apple-ios/debug/deps/libscrap_ffi-01ed18e308d8ab79.a
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
warning: output filename collision.
The lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)` has the same output filename as the lib target `scrap_ffi` in package `scrap-ffi v0.1.0 (/Users/nologik/satori.net/flutterust/native/scrap-ffi)`.
Colliding filename is: /Users/nologik/satori.net/flutterust/target/aarch64-apple-ios/debug/libscrap_ffi.a
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.
thread 'main' panicked at 'assertion failed: mtimes.insert(output.clone(), mtime).is_none()', src/tools/cargo/src/cargo/core/compiler/fingerprint.rs:996:13
stack backtrace:
   0:        0x10100a1d4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hb0278541eabdb65b
   1:        0x101029cdc - core::fmt::write::h6eab26c41e1615a6
   2:        0x1010026b0 - std::io::Write::write_fmt::h53012594e7e61aa3
   3:        0x10100ca70 - std::panicking::default_hook::{{closure}}::h3d38db2d154383b2
   4:        0x10100c598 - std::panicking::default_hook::hb9d33e9315d46c05
   5:        0x10100d024 - std::panicking::rust_panic_with_hook::hcf65477e7471a4ca
   6:        0x10100cbd8 - std::panicking::begin_panic_handler::{{closure}}::h98a6978c37ed787f
   7:        0x10100a68c - std::sys_common::backtrace::__rust_end_short_backtrace::h989ccc6ff53356f4
   8:        0x10100cb68 - _rust_begin_unwind
   9:        0x10107d894 - core::panicking::panic_fmt::h72cacefe63e11063
  10:        0x10107d820 - core::panicking::panic::h0416c1ef77139eda
  11:        0x100b173c4 - cargo::core::compiler::fingerprint::calculate::ha7ebd2e8d5375aa7
  12:        0x100b11bec - cargo::core::compiler::fingerprint::prepare_target::h9e061c15f1b23525
  13:        0x100c66b5c - cargo::core::compiler::compile::h8e1a5b2363291b32
  14:        0x1009edb5c - cargo::core::compiler::context::Context::compile::hf9bb50277194baa7
  15:        0x100bcc468 - cargo::ops::cargo_compile::compile_ws::h9ba552f8d250f9be
  16:        0x100bcc1e0 - cargo::ops::cargo_compile::compile::h169304b304473a16
  17:        0x100954404 - cargo::commands::build::exec::h99c42455ffd71d1e
  18:        0x10096a340 - cargo::cli::main::h109414b41770b785
  19:        0x10096394c - cargo::main::hb4945485930fa182
  20:        0x10095ce00 - std::sys_common::backtrace::__rust_begin_short_backtrace::h31ccbcc83be8cd7a
  21:        0x10095cfcc - std::rt::lang_start::{{closure}}::h2b1de5896d1ddb85
  22:        0x10100d46c - std::rt::lang_start_internal::hfcf8d8a4320307b0
  23:        0x1009657a4 - _main
[ERROR cargo_lipo] Failed to build "scrap-ffi" for "aarch64-apple-ios": Executing "/Users/nologik/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/cargo" "--color" "auto" "build" "-p" "scrap-ffi" "--target" "aarch64-apple-ios" "--lib" finished with error status: exit code: 101
[cargo-make][1] ERROR - Error while executing command, exit code: 1
[cargo-make][1] WARN - Build Failed.
[cargo-make] ERROR - Error while executing command, exit code: 1
[cargo-make] WARN - Build Failed.

Reguardless, I wanted to do what you recommended me to try. When I do a fresh git clone of flutterust, then run cargo make, android compiles fine, but ios returns an error:

Finished dev [unoptimized + debuginfo] target(s) in 0.61s
[INFO  cargo_lipo::lipo] Creating universal library for scrap-ffi
fatal error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: archive member /Users/nologik/tmp/flutterust/target/x86_64-apple-ios/debug/libscrap_ffi.a(aesni-x86_64-macosx.o) cputype (16777228) and cpusubtype (0) does not match previous archive members cputype (16777223) and cpusubtype (3) (all members must match)
[ERROR cargo_lipo] Executing "lipo" "-create" "-output" "/Users/nologik/tmp/flutterust/target/universal/debug/libscrap_ffi.a" "/Users/nologik/tmp/flutterust/target/aarch64-apple-ios/debug/libscrap_ffi.a" "/Users/nologik/tmp/flutterust/target/x86_64-apple-ios/debug/libscrap_ffi.a" finished with error status: exit code: 1
[cargo-make][1] ERROR - Error while executing command, exit code: 1
[cargo-make][1] WARN - Build Failed.
[cargo-make] ERROR - Error while executing command, exit code: 1
[cargo-make] WARN - Build Failed.

Finally, it appears that cargo automatically selects the appropriate lib type, so, there may be no need for having crate-type = ["rlib"], and instead, using crate-type = ["staticlib", "cdylib", "rlib"]. This would mean that the Makefile.toml no longer needs the restore-crate-type (and its counterpart) subroutine (as evidenced by cargo's output here:

[INFO  cargo_lipo::lipo] Building "scrap-ffi" for "x86_64-apple-ios"
   Compiling scrap-ffi v0.1.0 (/Users/nologik/tmp/flutterust/native/scrap-ffi)
warning: dropping unsupported crate type `cdylib` for target `x86_64-apple-ios`

warning: 1 warning emitted

)

shekohex commented 3 years ago

That's a progress, I guess you will also need to update cargo-lipo version too.

and besides these warnings, the script that restore the crate type is used to make/force android side to only compile a shared library, not a static one.

and the other way around in iOS, since in Android or at least in flutter, we really don't need the static library, so that cuts the compile time for Android a lot!

tbraun96 commented 3 years ago

It appears the trick is just to selectively compile targets instead of just running cargo make to save time.

For the record, cargo-lipo is also the latest version, so maybe there's just extra friction when dealing with a mac host (apple silicon)

shekohex commented 3 years ago

oh, you on the new apple m1 chips, okay feel free to keep this open, and it would be very helpful if you got time to write this up, so it could help others. (also feel free to open a PR to the README, I'll Happily accept and merge them).

Thanks!

alexmay23 commented 2 years ago

Hey. did you figure out how to resolve the issue with lipo and Mac ARM CPU ?