search

dart-lang / sdk

The Dart SDK, including the VM, JS and Wasm compilers, analysis, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
10.12k stars 1.57k forks source link

Reproducible Builds #55282

Open jwinarske opened 6 months ago

jwinarske commented 6 months ago 
#### General info

- Dart 3.3.1 (stable) (Wed Mar 6 13:09:19 2024 +0000) on "linux_x64"
- on linux / Linux 6.7.9-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Mar  6 19:35:04 UTC 2024
- locale is en_US.UTF-8

#### Process info

No Dart processes found.

This issue is about reproducible builds. The binary artifact libapp.so should only contain relative paths.

Build Example

kernel_snapshot_release step

dart 
--disable-analytics 
--disable-dart-dev
/mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/recipe-sysroot-native/usr/share/flutter/sdk/bin/cache/artifacts/engine/linux-x64/frontend_server.dart.snapshot 
--sdk-root /mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/recipe-sysroot-native/usr/share/flutter/sdk/bin/cache/artifacts/engine/common/flutter_patched_sdk_product/ 
--target=flutter 
--no-print-incremental-dependencies 
-Ddart.vm.profile=false 
-Ddart.vm.product=true 
--aot 
--tfa 
--target-os linux 
--packages .dart_tool/package_config.json 
--output-dill .dart_tool/flutter_build/*/app.dill 
--depfile .dart_tool/flutter_build/*/kernel_snapshot.d 
--source .dart_tool/flutter_build/dart_plugin_registrant.dart 
--source package:flutter/src/dart_plugin_registrant.dart 
-Dflutter.dart_plugin_registrant=.dart_tool/flutter_build/dart_plugin_registrant.dart 
--verbosity=error
package:gallery/main.dart

aot_elf_release step

gen_snapshot 
--deterministic 
--snapshot_kind=app-aot-elf 
--elf=libapp.so 
--strip 
.dart_tool/flutter_build/*/app.dill

Output

$ strings ./git/libapp.so |grep tmp
file:///mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/git/.dart_tool/flutter_build/dart_plugin_registrant.dart

Due to this absolute path the build is not reproducible. If this was built on a different machine, it would have a different path embedded in the shared module.

.dart_tool/flutter_build/dart_plugin_registrant.dart has @pragma('vm:entry-point') defined.

srawlins commented 6 months ago

Tentatively triaging to VM. Let me know if this is more of a language issue...

alexmarkov commented 6 months ago

This happens because Flutter generates .dart_tool/flutter_build/dart_plugin_registrant.dart file outside of a package. It has file:// URI which includes absolute path. This is a Flutter tooling issue, consider filing it in the Flutter repository.

Also, it is not clear why Flutter needs both:

--source .dart_tool/flutter_build/dart_plugin_registrant.dart 
--source package:flutter/src/dart_plugin_registrant.dart
jwinarske commented 6 months ago

@alexmarkov I get the same behavior regardless of the combo passed; with or without file:// prefix. app.dill as source to gen_snapshot contains absolute paths for all of the dart files.

--source .dart_tool/flutter_build/dart_plugin_registrant.dart 
--source package:flutter/src/dart_plugin_registrant.dart

These two files are not the same. The local folder one is what shows up in libapp.so post gen_snapshot:

$ strings libapp.so |grep tmp
file:///mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/git/.dart_tool/flutter_build/dart_plugin_registrant.dart

The problem lies in gen_snapshot.

alexmarkov commented 6 months ago

Compiler retains URIs as they may be needed at runtime (for example to show stack traces). This is not a problem for package:// URIs, but file:// URIs include absolute paths. In this case, Flutter tooling added .dart_tool/flutter_build/dart_plugin_registrant.dart to the program being compiled. This file lives outside of any package, so compiler includes its file:// URI.

Maybe you can try using --obfuscate and --split-debug-info Flutter options (see https://docs.flutter.dev/deployment/obfuscate) to avoid those absolute paths.

jwinarske commented 6 months ago

I see. Needing the path for debugging makes sense, only the file is transient. It would only resolve if the image was built. The other scenario is there are no debug tools in the case of AOT.

Neither of these (--obfuscate --strip / --obfuscate --save-obfuscation-map=obfuscation.map --strip) impact the issue:

$ gen_snapshot --deterministic --obfuscate --strip --snapshot_kind=app-aot-elf --elf=libapp.so --strip .dart_tool/flutter_build/*/app.dill
$ strings libapp.so |grep tmp
file:///mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/git/.dart_tool/flutter_build/dart_plugin_registrant.dart
$ gen_snapshot --deterministic --obfuscate --save-obfuscation-map=obfuscation.map --strip --snapshot_kind=app-aot-elf --elf=libapp.so --strip .dart_tool/flutter_build/*/app.dill
$ strings libapp.so |grep tmp
file:///mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/git/.dart_tool/flutter_build/dart_plugin_registrant.dart

I'm open to maintaining a patch. Just not clear what to patch in gen_snapshot.

jwinarske commented 6 months ago

This combo looks promising:

$ dart --disable-analytics --disable-dart-dev /mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/recipe-sysroot-native/usr/share/flutter/sdk/bin/cache/artifacts/engine/linux-x64/frontend_server.dart.snapshot --sdk-root /mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/recipe-sysroot-native/usr/share/flutter/sdk/bin/cache/artifacts/engine/common/flutter_patched_sdk_product/ --target=flutter --no-print-incremental-dependencies -Ddart.vm.profile=false -Ddart.vm.product=true --aot --tfa --target-os linux --packages .dart_tool/package_config.json  --output-dill .dart_tool/flutter_build/*/app.dill  --depfile .dart_tool/flutter_build/*/kernel_snapshot.d --source package:flutter/src/dart_plugin_registrant.dart  -Dflutter.dart_plugin_registrant=.dart_tool/flutter_build/dart_plugin_registrant.dart --verbosity=error package:gallery/main.dart
result c38bebdc-5ba9-4c1d-b1ab-8ebc1b318c9e
c38bebdc-5ba9-4c1d-b1ab-8ebc1b318c9e
c38bebdc-5ba9-4c1d-b1ab-8ebc1b318c9e .dart_tool/flutter_build/b7af51db6b4c9bfe7a0b7902f9856b16/app.dill 0
$ ../git/engine_sdk/sdk/clang_x64/gen_snapshot --deterministic --obfuscate --save-obfuscation-map=obfuscation.map --strip --snapshot_kind=app-aot-elf --elf=libapp.so --strip .dart_tool/flutter_build/*/app.dill
Warning: Generating ELF library without DWARF debugging information.
joel@air:/mnt/raid10/yocto/master/raspberry-pi5/tmp/work/cortexa76-poky-linux/flutter-gallery/2.10.2/git$ strings libapp.so |grep tmp

Contents of packages/flutter/lib/src/dart_plugin_registrant.dart

$ cat ./recipe-sysroot-native/usr/share/flutter/sdk/packages/flutter/lib/src/dart_plugin_registrant.dart
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

/// The location of the Dart Plugin Registrant. This is used by the engine to
/// execute the Dart Plugin Registrant when the Isolate is started or
/// DartPluginRegistrant.ensureInitialized() is called from a background
/// Isolate.
@pragma('vm:entry-point')
const String dartPluginRegistrantLibrary = String.fromEnvironment('flutter.dart_plugin_registrant');

How does one confirm the entry point other than via a target test?

alexmarkov commented 6 months ago

If you're running gen_snapshot directly, then you can also try its --dwarf-stack-traces option. Flutter tool passes it when --split-debug-info is specified.

jwinarske commented 6 months ago

I see dartPluginRegistrantLibrary in the data.debug output but it's not clear it's the actual vm entry point. How do I determine the vm:entry-point?

Child 109170 (at offset 0x4ff649):
  Abbreviation code: 3):
    DW_AT_abstract_origin => 0x000fa89d (origin: dartPluginRegistrantLibrary)
    DW_AT_low_pc => 0x10dbf44
    DW_AT_high_pc => 0x10dbf4c
    DW_AT_artificial => false
...
Debug information entry at offset 0x4ff649:
Abbreviation code: 3):
  DW_AT_abstract_origin => 0x000fa89d (origin: dartPluginRegistrantLibrary)
  DW_AT_low_pc => 0x10dbf44
  DW_AT_high_pc => 0x10dbf4c
  DW_AT_artificial => false
jwinarske commented 6 months ago

With my last idea I stated looked promising, testing on device dartPluginRegistrantLibrary is not called. It would seem only the implicit file:// addition works, which leaves the problem as I originally documented.

a-siva commented 6 months ago

@jwinarske are you using 'flutter tools' to build the app or do you have your own set of scripts for generating the final app ?

jwinarske commented 6 months ago

I manually cross compile (Yocto) using this function: https://github.com/meta-flutter/meta-flutter/blob/master/conf/include/common.inc#L283

I tried to setup a minimum viable repro for the flutter team this morning, only not cross compiling with standard flutter bits does not repro.

I custom build the flutter engine, and use the gen_snapshot from this. Linux_simarm, linux_simarm64, linux_simx86, linux_x64, linux_simriscv32, and linux_simriscv64 are used depending on the BSP/machine architecture. The buildroot flutter work is all based/derived from my work as well.

Not sure yet if it's isolated to linux_sim...

a-siva commented 5 months ago

@jwinarske without a minimal repro case for us to debug/investigate there is not much we can do here, please provide steps for a reproducing the problem.

jwinarske commented 5 months ago

@a-siva The initial post here has the steps to repro. The dart-sdk gen_snapshot would need to be linux_simarm64. Anything other than a straight host build; which is what the flutter tooling uses.