[Impeller] Can only build the host artifacts if disabling unit tests in master channel.

[erickzanardo]

Is there an existing issue for this?

• [X] I have searched the existing issues
• [X] I have read the guide to filing a bug

Steps to reproduce

1. Follow https://github.com/flutter/flutter/wiki/Setting-up-the-Engine-development-environment
2. Follow https://github.com/flutter/flutter/wiki/Compiling-the-engine
   • Try to compile the host artifacts by doing ./flutter/tools/gn --enable-impeller-3d --no-lto and then ninja -C out/host_debug

Expected results

host artifacts builds without errors

Actual results

ninja commands breaks with the following error:

ninja -C out/host_debug
ninja: Entering directory `out/host_debug'
[1143/6269] ACTION //flutter/impeller/ren...fixtures(//build/toolchain/mac:clang_x64)
FAILED: gen/flutter/impeller/renderer/assets/kernel_blob.bin
vpython3 ../../build/gn_run_binary.py ../../flutter/prebuilts/macos-x64/dart-sdk/bin/dart /Users/erickzanardo/projects/flutter_impeller_2/src/flutter/prebuilts/macos-x64/dart-sdk/bin/snapshots/frontend_server.dart.snapshot --sdk-root /Users/erickzanardo/projects/flutter_impeller_2/src/out/host_debug/flutter_patched_sdk --target flutter --sound-null-safety --output-dill gen/flutter/impeller/renderer/assets/kernel_blob.bin --depfile /Users/erickzanardo/projects/flutter_impeller_2/src/out/host_debug/gen/flutter/impeller/renderer/snapshot__dsk__ds_renderer_dart_fixtures.depfile.d /Users/erickzanardo/projects/flutter_impeller_2/src/flutter/impeller/fixtures/dart_tests.dart
result 4c0208b3-f839-469c-aab9-c452dfd95883
../../flutter/lib/gpu/lib/src/smoketest.dart:24:1: Error: The 'class-modifiers' language feature is disabled for this library.
Try removing the package language version or setting the language version to 3.0 or higher.
base class FlutterGpuTestClass extends NativeFieldWrapperClass1 {
^^^^
../../flutter/lib/gpu/lib/src/buffer.dart:38:1: Error: The 'class-modifiers' language feature is disabled for this library.
Try removing the package language version or setting the language version to 3.0 or higher.
base class HostBuffer extends NativeFieldWrapperClass1 {
^^^^
4c0208b3-f839-469c-aab9-c452dfd95883
+file:///Users/erickzanardo/projects/flutter_impeller_2/src/flutter/impeller/fixtures/dart_tests.dart
+file:///Users/erickzanardo/projects/flutter_impeller_2/src/flutter/lib/gpu/lib/gpu.dart
+file:///Users/erickzanardo/projects/flutter_impeller_2/src/flutter/lib/gpu/lib/src/smoketest.dart
+file:///Users/erickzanardo/projects/flutter_impeller_2/src/flutter/lib/gpu/lib/src/context.dart
+file:///Users/erickzanardo/projects/flutter_impeller_2/src/flutter/lib/gpu/lib/src/buffer.dart
4c0208b3-f839-469c-aab9-c452dfd95883 gen/flutter/impeller/renderer/assets/kernel_blob.bin 2

[1150/6269] ACTION //flutter/shell/common...fixtures(//build/toolchain/mac:clang_x64)

If I try running again, with --no-enable-unittests flag, the process will succeed without any issues.

Code sample

Since this is in the engine, I don't have any code sample, but I've been able to track down the issue to this command that is run by ninja:

flutter/prebuilts/macos-x64/dart-sdk/bin/dart /Users/erickzanardo/projects/flutter_impeller_2/src/flutter/prebuilts/macos-x64/dart-sdk/bin/snapshots/frontend_server.dart.snapshot --sdk-root /Users/erickzanardo/projects/flutter_impeller_2/src/out/host_debug/flutter_patched_sdk --target flutter --sound-null-safety --output-dill gen/flutter/impeller/renderer/assets/kernel_blob.bin --depfile /Users/erickzanardo/projects/flutter_impeller_2/src/out/host_debug/gen/flutter/impeller/renderer/snapshot__dsk__ds_renderer_dart_fixtures.depfile.d /Users/erickzanardo/projects/flutter_impeller_2/src/flutter/impeller/fixtures/dart_tests.dart

If I manually run it, I get the same output:

result caa56d37-f5ad-4114-b1f2-7020ab8bb7bb
flutter/lib/gpu/lib/src/smoketest.dart:24:1: Error: The 'class-modifiers' language feature is disabled for this library.
Try removing the package language version or setting the language version to 3.0 or higher.
base class FlutterGpuTestClass extends NativeFieldWrapperClass1 {
^^^^
flutter/lib/gpu/lib/src/buffer.dart:38:1: Error: The 'class-modifiers' language feature is disabled for this library.
Try removing the package language version or setting the language version to 3.0 or higher.
base class HostBuffer extends NativeFieldWrapperClass1 {
^^^^
Unhandled exception:
PathNotFoundException: Cannot open file, path = 'gen/flutter/impeller/renderer/assets/kernel_blob.bin' (OS Error: No such file or directory, errno = 2)
#0      _checkForErrorResponse (dart:io/common.dart:55:9)
#1      _File.open.<anonymous closure> (dart:io/file_impl.dart:381:7)
<asynchronous suspension>
#2      _FileStreamConsumer.addStream.<anonymous closure> (dart:io/file_impl.dart:205:22)
<asynchronous suspension>

Screenshots or Video

Screenshots / Video demonstration

[Upload media here]

Logs

Logs

```console [Paste your logs here] ```

Flutter Doctor output

Doctor output

```console [✓] Flutter (Channel master, 3.14.0-14.0.pre.358, on macOS 13.5 22G74 darwin-arm64, locale en-GB) • Flutter version 3.14.0-14.0.pre.358 on channel master at /Users/erickzanardo/flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision 8936504163 (2 days ago), 2023-09-19 02:10:24 -0400 • Engine revision 589bde9a95 • Dart version 3.2.0 (build 3.2.0-174.0.dev) • DevTools version 2.28.0-dev.8 [!] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/erickzanardo/Library/Android/sdk • Platform android-33, build-tools 31.0.0 • ANDROID_HOME = /Users/erickzanardo/Library/Android/sdk • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java ✗ Cannot execute /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java to determine the version [✓] Xcode - develop for iOS and macOS (Xcode 14.3.1) • Xcode at /Applications/Xcode.app/Contents/Developer • Build 14E300c • CocoaPods version 1.12.1 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [!] Android Studio (not installed) • Android Studio not found; download from https://developer.android.com/studio/index.html (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions). [✓] Connected device (2 available) • macOS (desktop) • macos • darwin-arm64 • macOS 13.5 22G74 darwin-arm64 • Chrome (web) • chrome • web-javascript • Google Chrome 116.0.5845.187 [✓] Network resources • All expected network resources are available. ! Doctor found issues in 2 categories. ```

[chinmaygarde]

I am able to build just fine at ToT (currently at 29d3c89486cacb830e253e45e92af243d095bafb). The notice about "language version to 3.0 or higher" indicates that perhaps a gclient sync -D hasn't happened. Can you checkout that revision and run gclient sync -D and try again?

[erickzanardo]

I am able to build just fine at ToT (currently at 29d3c89486cacb830e253e45e92af243d095bafb). The notice about "language version to 3.0 or higher" indicates that perhaps a gclient sync -D hasn't happened. Can you checkout that revision and run gclient sync -D and try again?

@chinmaygarde I've tried gclient sync -D a couple of times. Just to be sure I've tried again after I checked out 29d3c89486cacb830e253e45e92af243d095bafb, but the error "language version to 3.0 or higher" persists.

This was the steps I took, please let me know if I did something wrong:

# Inside src/flutter
git checkout 29d3c89486cacb830e253e45e92af243d095bafb

# inside src
gclient sync -D
./flutter/tools/gn --enable-impeller-3d --no-lto --mac-cpu=arm64
ninja -C out/host_debug_arm64

Thanks!

[chinmaygarde]

cc @bdero Are you able to reason about the errors in the Flutter GPU Dart code? I am not able to reproduce on an M1 Max.

[bdero]

On Discord I asked @erickzanardo to file this bug with all the deets because I was also unable to reproduce it, unfortunately. The way I'm importing Flutter GPU files in the test fixture is a bit haphazard and perhaps we should be building the snapshot with a package configuration similar to Flutter tester.

Also, there used to be a way to specify the dart version inline in a file, but I think we removed use of that all over the engine repo after deprecation?

[erickzanardo]

@chinmaygarde @bdero, I was trying many stuff to try to make the compilation work, and one thought that I had was to remove all .dart* folder from my home!

After I did that, I was able to compile the engine without the need to the --no-enable-unittests.

[chinmaygarde]

@bdero We should rework the Flutter GPU tests like we do the lib/ui unit-tests. I am not sure why we still have this class of issues.

[chinmaygarde]

Closing this as there isn't any immediate action item. We'll be reworking the test harness to work like ui_unittests and embedder_unittests in the future and this issue should be moot then.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

[goderbauer]

I am running into the exact same issue. I tried from a clean checkout of the engine and the problem persist. I currently cannot compile the unit tests locally. This is on an Intel Mac.

[erickzanardo]

@goderbauer in my case I was to build after I deleted the '.dart' folder in the home of my computer, after that the build with the unit tests started to work again

[goderbauer]

Interesting. I tried that, but still no luck. Same error.

[goderbauer]

The solution to the mystery is: I had an outdated .dart_tool/package_config.json in the directory that contained my engine checkout (not my home directory). After deleting that one, everything works again. I wasn't aware that the dart vm would just walk up the entire directory tree and use whatever .dart_tool/package_config.json it can find.

Maybe we should add some tooling to our gclient sync -D invocation that checks for stray .dart_tool/package_config.json up the directory tree and warns about those...

[erickzanardo]

The solution to the mystery is: I had an outdated .dart_tool/package_config.json in the directory that contained my engine checkout (not my home directory). After deleting that one, everything works again. I wasn't aware that the dart vm would just walk up the entire directory tree and use whatever .dart_tool/package_config.json it can find.

Maybe we should add some tooling to our gclient sync -D invocation that checks for stray .dart_tool/package_config.json up the directory tree and warns about those...

Urrhg, now that you said that, I believe the command that I ran was actually rm -rf ~/.dart*, so I believe that maybe the solution was deleting the .dart_tool instead. Sorry for that, my memory is not what it used to be.

Adding tooling to warn about that seems like a good idea! cause although it seems that this issue is very rare and apparently easy to fix, it is very not really straight forward to come up to its realisation.

[jonahwilliams]

We've removed scene from being a special engine recipie which was the underlying cause of many of these issues.