search

sony / flutter-embedded-linux

Embedded Linux embedding for Flutter
BSD 3-Clause "New" or "Revised" License
1.17k stars 124 forks source link

Test on arm devices #4

Open HidenoriMatsubayashi opened 3 years ago

HidenoriMatsubayashi commented 3 years ago

This issue is a memo for testing on different arm devices. I would like to test on a lot of arm devices.

Tested devices

HidenoriMatsubayashi commented 2 years ago

@atornqvist

Could you try the following?

$ export CC=${CLANGCC}
$ export CXX=${CLANGCXX}
$ flutter-elinux create exampleapp
$ cd exampleapp
$ flutter-elinux build elinux --target-arch=arm64

See also: https://github.com/sony/meta-flutter#cross-building-using-yocto-sdk

atornqvist commented 2 years ago

@HidenoriMatsubayashi

Thanks for the response.

Unfortunately it makes no difference. The linking still goes wrong.

HidenoriMatsubayashi commented 2 years ago

@atornqvist

=====> Linker error since it is the Yocto x86_64 ld that is used and not the cross-build ld.

Could you inform me of the error messages?

atornqvist commented 2 years ago

@HidenoriMatsubayashi . Yes, sure. Below is the last part. It seems like the flutter build is not using the cross clang++ and then that leads to the error with ld.

CMake Error at
/scratch/opt/poky/3.1.15/sysroots/x86_64-pokysdk-linux/usr/share/cmake-3.16/Modules/CMakeTestCXXComp
iler.cmake:53 (message):
  The C++ compiler

    "/scratch/opt/poky/3.1.15/sysroots/x86_64-pokysdk-linux/usr/bin/clang++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/anders/flutter_tests/exampleapp/build/elinux/arm64/release/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/make cmTC_5e7f1/fast && /usr/bin/make -f
    CMakeFiles/cmTC_5e7f1.dir/build.make CMakeFiles/cmTC_5e7f1.dir/build
    make[1]: Entering directory
    '/home/anders/flutter_tests/exampleapp/build/elinux/arm64/release/CMakeFiles/CMakeTmp'
    Building CXX object CMakeFiles/cmTC_5e7f1.dir/testCXXCompiler.cxx.o
    /scratch/opt/poky/3.1.15/sysroots/x86_64-pokysdk-linux/usr/bin/clang++
    --target=aarch64-poky-linux --sysroot=/scratch/opt/poky/3.1.15/sysroots/aarch64-poky-linux
    -O2 -pipe -g -feliminate-unused-debug-types     -o
    CMakeFiles/cmTC_5e7f1.dir/testCXXCompiler.cxx.o -c
    /home/anders/flutter_tests/exampleapp/build/elinux/arm64/release/CMakeFiles/CMakeTmp/testCXXComp
    iler.cxx
    Linking CXX executable cmTC_5e7f1
    /scratch/opt/poky/3.1.15/sysroots/x86_64-pokysdk-linux/usr/bin/cmake -E cmake_link_script
    CMakeFiles/cmTC_5e7f1.dir/link.txt --verbose=1
    /scratch/opt/poky/3.1.15/sysroots/x86_64-pokysdk-linux/usr/bin/clang++
    --target=aarch64-poky-linux --sysroot=/scratch/opt/poky/3.1.15/sysroots/aarch64-poky-linux   -O2
    -pipe -g -feliminate-unused-debug-types    -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed
    -fstack-protector-strong -Wl,-z,relro,-z,now  CMakeFiles/cmTC_5e7f1.dir/testCXXCompiler.cxx.o
    -o cmTC_5e7f1
    /scratch/opt/poky/3.1.15/sysroots/x86_64-pokysdk-linux/usr/bin/ld: unrecognised emulation mode:
    aarch64linux
    Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu elf_l1om elf_k1om i386pep
    i386pe
    clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
    make[1]: *** [CMakeFiles/cmTC_5e7f1.dir/build.make:87: cmTC_5e7f1] Error 1
    make[1]: Leaving directory
    '/home/anders/flutter_tests/exampleapp/build/elinux/arm64/release/CMakeFiles/CMakeTmp'
    make: *** [Makefile:121: cmTC_5e7f1/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)
topolittle commented 2 years ago

Here is what I done to patch the Yocto's SDK to make it works with flutter-elinux. After installing the SDK, I have done the following:

export TARGET_ARCH="aarch64-poky-linux" && \
. /opt/fsl-imx-wayland/5.4-zeus/environment-setup-$TARGET_ARCH && \
echo 'set ( CMAKE_C_COMPILER_WORKS 1 )' >> $OECORE_NATIVE_SYSROOT/usr/share/cmake/OEToolchainConfig.cmake && \
echo 'set ( CMAKE_C_COMPILER_TARGET "'$TARGET_ARCH'" )' >> $OECORE_NATIVE_SYSROOT/usr/share/cmake/OEToolchainConfig.cmake && \
echo 'set ( CMAKE_CXX_COMPILER_WORKS 1 )' >> $OECORE_NATIVE_SYSROOT/usr/share/cmake/OEToolchainConfig.cmake && \
echo 'set ( CMAKE_CXX_COMPILER_TARGET "'$TARGET_ARCH'" )' >> $OECORE_NATIVE_SYSROOT/usr/share/cmake/OEToolchainConfig.cmake && \
mv $OECORE_NATIVE_SYSROOT/usr/bin/ld $OECORE_NATIVE_SYSROOT/usr/bin/ld.x86_64 && \
cp -a $OECORE_NATIVE_SYSROOT/usr/bin/$TARGET_ARCH/${TARGET_PREFIX}ld $OECORE_NATIVE_SYSROOT/usr/bin/ld

In a nutshell, it disables the compiler test for C and C++, it sets the target architecture and it swaps the x86_64 and the ARM64 linker since the SDK is using the wrong linker.

I am running with this fix for months without any issues.

HidenoriMatsubayashi commented 2 years ago

@topolittle

Thank you very much. I'd like to add the doc how to build with Yocto SDK in flutter-elinux. Could you summarize how to build? I'll copy and paste your document into here.

LefixBlue commented 2 years ago

Hi @HidenoriMatsubayashi @topolittle @atornqvist ,

as far as i remember (I built everything for/with imx-yocto zeus release 2 months ago) I just had to patch flutter-elinux to match the compiler-triple used in Yocto:

The file: flutter-elinux/lib/elinux_build_target.dart

The patched two lines: '-DCMAKE_C_COMPILER_TARGET=aarch64-poky-linux', '-DCMAKE_CXX_COMPILER_TARGET=aarch64-poky-linux',

And ensure a rebuild of flutter-elinux afterwards!

Then set the Yocto-SDK environment and build the application with flutter-elinux

I havn't tried it with yocto-dunfell yet. But I wonder what could cause this issue as it was really that easy when using yocto-zeus release.

So the time I built it, I thought it would be enough to let flutter-elinux allow to give the compiler-triple (in our case: 'aarch64-poky-linux') as a cmd-line argument.

HidenoriMatsubayashi commented 2 years ago

@LefixBlue Thank you for your comment.

So the time I built it, I thought it would be enough to let flutter-elinux allow to give the compiler-triple (in our case: 'aarch64-poky-linux') as a cmd-line argument.

I'll create a patch! Please watch https://github.com/sony/flutter-elinux/issues/48

HidenoriMatsubayashi commented 2 years ago

Now, I added target-compiler-triple option for cross-building. https://github.com/sony/flutter-elinux/pull/88

LefixBlue commented 2 years ago

@HidenoriMatsubayashi

That looks good to me!

I will try the new option in my environment as soon as I'll have some free time.

atornqvist commented 2 years ago

@HidenoriMatsubayashi @LefixBlue

Looks good.

I made a quick test. It does not make any difference in my build.

What makes it build is the changing of the ld as @topolittle suggest with:

mv $OECORE_NATIVE_SYSROOT/usr/bin/ld $OECORE_NATIVE_SYSROOT/usr/bin/ld.x86_64 && \
cp -a $OECORE_NATIVE_SYSROOT/usr/bin/$TARGET_ARCH/${TARGET_PREFIX}ld $OECORE_NATIVE_SYSROOT/usr/bin/ld

What I can see in the build is that it is "clang++" that is used and not "aarch64-poky-linux-clang++". Then it will be the clang++ in the SDK directory that will used and that uses the x86_64 "ld".

topolittle commented 2 years ago

@atornqvist @HidenoriMatsubayashi In the Yocto SDK, the cross-compiler are in the /usr/bin// and the native compilers are in /usr/bin/

For the triplets, is not possible to move the clang compiler binaries like I do for the linker. The compiler in both folder are the same binaries and when invoked, they behave according to their filename, somewhat like busybox do. Setting CMAKE_C_COMPILER_TARGET and CMAKE_CXX_COMPILER_TARGET in the toolchain cmake configuration file make it use the right compiler path and filename.

atornqvist commented 2 years ago

Hi, the compiler triplet does not seem to be used for CXX since flutter-elinux/lib/elinux_build_target.dart line"environment: <String, String>{'CC': 'clang', 'CXX': 'clang++'}" sets it to "clang++".

Deleteing that line makes the cross build work fine with Poky dunfell.

No changes in the toolchain cmake file is needed as long as the environment is set with this: export CC=${CLANGCC} export CXX=${CLANGCXX}

As reference the situation can be reproduced with:

# Build a dunfell Poky SDK:
git clone git://git.yoctoproject.org/poky.git -b dunfell
pushd poky && git checkout 64f632c93f487004e721ec7b4ae61b817157037b && popd
git clone https://github.com/kraj/meta-clang -b dunfell
pushd meta-clang && git checkout 3bb001d3f364bbf6588fed04b9ee2c7e74a0beba && popd
git clone https://github.com/sony/meta-flutter.git
pushd meta-flutter && git checkout 5818f865c5a7df2955b6de60bc1cdfd8b2ef7a2e && popd

source poky/oe-init-build-env build

cat >> conf/local.conf << EOF
MACHINE ?= "qemuarm64"
CLANGSDK = "1"
EOF

bitbake-layers add-layer ../meta-clang
bitbake-layers add-layer ../meta-flutter

bitbake core-image-weston -c populate_sdk

tmp/deploy/sdk/poky-glibc-x86_64-core-image-weston-aarch64-qemuarm64-toolchain-3.1.16.sh

export SDK_INSTALL_PATH=/opt/poky/3.1.16

git clone https://github.com/sony/flutter-elinux.git
pushd flutter-elinux && git checkout e3b9d8e9ca676f49ad7febdca49f54dbd2f462a8 && popd

export TARGET_ARCH="aarch64-poky-linux" && \
. ${SDK_INSTALL_PATH}/environment-setup-$TARGET_ARCH

# Remove flutter-elinux line that sets CXX to a native clang++
# Doing this makes the cross build work.
# Skip this line to reproduce the build error
sed -i "/environment: <String, String>{'CC': 'clang', 'CXX': 'clang++'},/d" flutter-elinux/lib/elinux_build_target.dart

export CC=${CLANGCC}
export CXX=${CLANGCXX}

export PATH="$PATH:`pwd`/flutter-elinux/bin"
flutter-elinux precache
flutter-elinux create exampleapp
cd exampleapp
flutter-elinux build elinux --target-arch=arm64 --target-compiler-triple=$TARGET_ARCH --target-sysroot=$SDK_INSTALL_PATH/sysroots/aarch64-poky-linux/ -v
HidenoriMatsubayashi commented 2 years ago

@atornqvist Perhaps, Case 2: Use Yocto SDK helps you.

atornqvist commented 2 years ago

The moving of ld is not required if the flutter-elinux/lib/elinux_build_target.dart is modified as above. Then the CXX environment variable can be used to specify the appropriate clang++ that contains the toolchain triple for cross-building which will result in a working usage of ld.

Masterxiechengxun commented 2 years ago

Thank you!!! I have the same problem as you.Executing the followings you mentioned solves my problem. $ sed -i "/environment: <String, String>{'CC': 'clang', 'CXX': 'clang++'},/d" /opt/flutter-elinux/lib/elinux_build_target.dart

gencay3535 commented 1 year ago

Has anyone ever tried flutter with stm32mp15 on kirkstone branch? I've been trying for days but could not able to build an sdk for my flutter application.

justdoGIT commented 1 year ago

Hi @ALL while running flutter app in imx8m mini I am getting this error:

root@imx8mm-var-dart:~/my_app# flutter-client -b /home/root/my_app/build/elinux/arm64/release/bundle/ [FATAL:flutter/runtime/dart_vm_initializer.cc(22)] Error while initializing the Dart VM: Wrong full snapshot version, expected '1441d6b13b8623fa7fbf61433abebd31' found 'b0e899ec5a90e4661501f0b69e9dd70f' Aborted

Flutter version

root@imx8mm-var-dart:~/my_app# flutter-elinux run -d elinux-wayland --release Woah! You appear to be trying to run flutter as root. We strongly recommend running the flutter tool without superuser privileges. / 📎 Flutter 3.3.1 • channel unknown • unknown source Framework • revision 4f9d92fbbd (5 days ago) • 2022-09-06 17:54:53 -0700 Engine • revision 3efdf03e73 Tools • Dart 2.18.0 • DevTools 2.15.0 Launching lib/main.dart on eLinux in release mode... Building an eLinux application with wayland backend in release mode for arm64 target... 245.3s

Flutter run key commands. h List all available interactive commands. c Clear the screen q Quit (terminate the application on the device).

Application finished.

HidenoriMatsubayashi commented 1 year ago

Please try:

$ rm -r <path_to_flutter-elinux>/flutter/bin/cache/
$ flutter doctor
justdoGIT commented 1 year ago

Hi, after removing the flutter/bin/cache directory and then running $ flutter doctor command I am facing the same issue. The output is as shown below:

root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin/cache# rm -rf * root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin/cache# ls root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin/cache# cd .. root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin# ls cache dart dart.bat flutter flutter.bat internal root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin# ./flutter doctor Woah! You appear to be trying to run flutter as root. We strongly recommend running the flutter tool without superuser privileges. / 📎 Downloading Linux arm64 Dart SDK from Flutter engine 3efdf03e73dd352873fc5a7e1c0de04e52863b3a... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 118M 100 118M 0 0 5799k 0 0:00:20 0:00:20 --:--:-- 9116k Building flutter tool... Downloading Material fonts... 2,108ms Downloading Gradle Wrapper... 50ms Downloading package sky_engine... 547ms Downloading flutter_patched_sdk tools... 1,617ms Downloading flutter_patched_sdk_product tools... 1,025ms Downloading linux-arm64 tools... 5.9s Downloading linux-arm64/font-subset tools... 799ms Doctor summary (to see all details, run flutter doctor -v): [!] Flutter (Channel unknown, 3.3.1, on FSLC Wayland with XWayland 3.1 (dunfell) 5.4.142+gb84afa992407, locale en_US) ! Flutter version 3.3.1 on channel unknown at /opt/flutter-elinux/flutter ! Upstream repository unknown [✗] Chrome - develop for the web (Cannot find Chrome executable at google-chrome) ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable. [✗] Linux toolchain - develop for Linux desktop ✗ ninja is required for Linux development. It is likely available from your distribution (e.g.: apt install ninja-build), or can be downloaded from https://github.com/ninja-build/ninja/releases ✗ GTK 3.0 development libraries are required for Linux development. They are likely available from your distribution (e.g.: apt install libgtk-3-dev) [!] Flutter IDE Support (No supported IDEs installed) [✓] Connected device (1 available) [✓] HTTP Host Availability

! Doctor found issues in 4 categories.

root@imx8mm-var-dart:~/my_app# flutter-client -b /home/root/my_app/build/elinux/arm64/release/bundle/ [FATAL:flutter/runtime/dart_vm_initializer.cc(22)] Error while initializing the Dart VM: Wrong full snapshot version, expected '1441d6b13b8623fa7fbf61433abebd31' found 'b0e899ec5a90e4661501f0b69e9dd70f' Aborted

Steps to reproduce:

  1. Create fsl-image-gui yocto image with clang and cmake.
  2. Clone flutter-elinux in arm64 board.
  3. run $ flutter-elinux devices
  4. run $ flutter-elinux create my app
  5. run $ flutter-elinux run -d elinux-wayland --release
  6. run $ flutter-client -b /home/root/my_app/build/elinux/arm64/release/bundle

Please let me know if anything else is required from my side.

justdoGIT commented 1 year ago

Hi, after removing the flutter/bin/cache directory and then running $ flutter doctor command I am facing the same issue. The output is as shown below:

root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin/cache# rm -rf * root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin/cache# ls root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin/cache# cd .. root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin# ls cache dart dart.bat flutter flutter.bat internal root@imx8mm-var-dart:/opt/flutter-elinux/flutter/bin# ./flutter doctor Woah! You appear to be trying to run flutter as root. We strongly recommend running the flutter tool without superuser privileges. / paperclip Downloading Linux arm64 Dart SDK from Flutter engine 3efdf03e73dd352873fc5a7e1c0de04e52863b3a... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 118M 100 118M 0 0 5799k 0 0:00:20 0:00:20 --:--:-- 9116k Building flutter tool... Downloading Material fonts... 2,108ms Downloading Gradle Wrapper... 50ms Downloading package sky_engine... 547ms Downloading flutter_patched_sdk tools... 1,617ms Downloading flutter_patched_sdk_product tools... 1,025ms Downloading linux-arm64 tools... 5.9s Downloading linux-arm64/font-subset tools... 799ms Doctor summary (to see all details, run flutter doctor -v): [!] Flutter (Channel unknown, 3.3.1, on FSLC Wayland with XWayland 3.1 (dunfell) 5.4.142+gb84afa992407, locale en_US) ! Flutter version 3.3.1 on channel unknown at /opt/flutter-elinux/flutter ! Upstream repository unknown [✗] Chrome - develop for the web (Cannot find Chrome executable at google-chrome) ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable. [✗] Linux toolchain - develop for Linux desktop ✗ ninja is required for Linux development. It is likely available from your distribution (e.g.: apt install ninja-build), or can be downloaded from https://github.com/ninja-build/ninja/releases ✗ GTK 3.0 development libraries are required for Linux development. They are likely available from your distribution (e.g.: apt install libgtk-3-dev) [!] Flutter IDE Support (No supported IDEs installed) [✓] Connected device (1 available) [✓] HTTP Host Availability

! Doctor found issues in 4 categories.

root@imx8mm-var-dart:~/my_app# flutter-client -b /home/root/my_app/build/elinux/arm64/release/bundle/ [FATAL:flutter/runtime/dart_vm_initializer.cc(22)] Error while initializing the Dart VM: Wrong full snapshot version, expected '1441d6b13b8623fa7fbf61433abebd31' found 'b0e899ec5a90e4661501f0b69e9dd70f' Aborted

Steps to reproduce:

  1. Create fsl-image-gui yocto image with clang and cmake.
  2. Clone flutter-elinux in arm64 board.
  3. run $ flutter-elinux devices
  4. run $ flutter-elinux create my app
  5. run $ flutter-elinux run -d elinux-wayland --release
  6. run $ flutter-client -b /home/root/my_app/build/elinux/arm64/release/bundle

Please let me know if anything else is required from my side.

Also while running $ flutter-elinux build elinuxI am getting these warnings/ errors as shown below:

root@imx8mm-var-dart:~/my_app# flutter-elinux build elinux
   Woah! You appear to be trying to run flutter as root.
   We strongly recommend running the flutter tool without superuser privileges.
  /
📎
Flutter 3.3.1 • channel unknown • unknown source
Framework • revision 4f9d92fbbd (6 days ago) • 2022-09-06 17:54:53 -0700
Engine • revision 3efdf03e73
Tools • Dart 2.18.0 • DevTools 2.15.0
Downloading linux-arm64/linux-arm64-flutter-gtk tools...            6.1s
Downloading linux-arm64-profile/linux-arm64-flutter-gtk tools...         4.2s
Downloading linux-arm64-release/linux-arm64-flutter-gtk tools...         3.4s
Downloading an artifact that may not be reachable in some environments (e.g. firewalled environments):
https://github.com/sony/flutter-embedded-linux/releases/download/3efdf03e73/elinux-common.zip
This should not have happened. This is likely a Flutter SDK bug. Please file an issue at
https://github.com/flutter/flutter/issues/new?template=1_activation.md
Downloading elinux-common tools...                                     ⣯Downloading elinux-common tools...                                    2,995ms
Downloading an artifact that may not be reachable in some environments (e.g. firewalled environments):
https://github.com/sony/flutter-embedded-linux/releases/download/3efdf03e73/elinux-arm64-debug.zip
This should not have happened. This is likely a Flutter SDK bug. Please file an issue at
https://github.com/flutter/flutter/issues/new?template=1_activation.md
Downloading elinux-arm64-debug tools...                                ⣯Downloading elinux-arm64-debug tools...                                  5.6s
Downloading an artifact that may not be reachable in some environments (e.g. firewalled environments):
https://github.com/sony/flutter-embedded-linux/releases/download/3efdf03e73/elinux-arm64-profile.zip
This should not have happened. This is likely a Flutter SDK bug. Please file an issue at
https://github.com/flutter/flutter/issues/new?template=1_activation.md
Downloading elinux-arm64-profile tools...                              ⣯Downloading elinux-arm64-profile tools...                                3.3s
Downloading an artifact that may not be reachable in some environments (e.g. firewalled environments):
https://github.com/sony/flutter-embedded-linux/releases/download/3efdf03e73/elinux-arm64-release.zip
This should not have happened. This is likely a Flutter SDK bug. Please file an issue at
https://github.com/flutter/flutter/issues/new?template=1_activation.md
Downloading elinux-arm64-release tools...                              ⣯Downloading elinux-arm64-release tools...                             2,304ms
Downloading an artifact that may not be reachable in some environments (e.g. firewalled environments):
https://github.com/sony/flutter-embedded-linux/releases/download/3efdf03e73/elinux-x64-debug.zip
This should not have happened. This is likely a Flutter SDK bug. Please file an issue at
https://github.com/flutter/flutter/issues/new?template=1_activation.md
Downloading elinux-x64-debug tools...                                  ⣯Downloading elinux-x64-debug tools...                                    5.9s
Downloading an artifact that may not be reachable in some environments (e.g. firewalled environments):
https://github.com/sony/flutter-embedded-linux/releases/download/3efdf03e73/elinux-x64-profile.zip
This should not have happened. This is likely a Flutter SDK bug. Please file an issue at
https://github.com/flutter/flutter/issues/new?template=1_activation.md
Downloading elinux-x64-profile tools...                                ⣯Downloading elinux-x64-profile tools...                                 12.6s
Downloading an artifact that may not be reachable in some environments (e.g. firewalled environments):
https://github.com/sony/flutter-embedded-linux/releases/download/3efdf03e73/elinux-x64-release.zip
This should not have happened. This is likely a Flutter SDK bug. Please file an issue at
https://github.com/flutter/flutter/issues/new?template=1_activation.md
Downloading elinux-x64-release tools...                                ⣯Downloading elinux-x64-release tools...                                  7.0s

💪 Building with sound null safety 💪

Building an eLinux application with wayland backend in release mode for arm64 target...        255.9s
â¡¿root@imx8mm-var-dart:~/my_app#

Let me know if these warnings are okay or not

justdoGIT commented 1 year ago

Hi, I was able to run the flutter application finally. The cause was flutter-engine was using dunfell branch while the flutter sdk was using master branch. So checkout 3.0.5 tag in flutter sdk resolved the issue for me