Closed Utopiah closed 8 months ago
Utopiah
commented
8 months ago Details I forgot :
fabien@fabien-CORSAIR-ONE-i160:~/Prototypes/wolvic/gecko-dev/firefox-121.0.1$ cat mozconfig
ac_add_options --enable-project=mobile/android
ac_add_options --target=aarch64 --enable-linker=lld
ac_add_options --enable-project=mobile/android
svillar
commented
8 months ago You better ask in Mozilla channels for Gecko build issues. That said I've never hit that mingw error which indeed is related to the windows build. Have you run the match bootstrap first?
Utopiah
commented
8 months ago Indeed I understand it’s a Gecko problem so didn’t want to bother you with it but thought it might be due to Wolvic patches or documented process.
I did run match bootstrap first.
I will try on another machine with the previous Ubuntu just in case.
svillar
commented
8 months ago Indeed I understand it’s a Gecko problem so didn’t want to bother you with it but thought it might be due to Wolvic patches or documented process.
Wolvic patches could cause build issues but not at that point, i.e. the build configuration. You could try without applying the patches just in case
No problem in asking, just that Mozillians have very likely hit similar problems in the past so you'd likely get more useful info from their forums/channels.
I did run match bootstrap first.
I will try on another machine with the previous Ubuntu just in case.
In my experience Gecko builds are unreliable. Minor changes in your config can easily break them. Also as it uses as much local libraries as it can, it's really difficult to get reproducible builds or even normal builds but from previous versions with different library dependencies.
Utopiah
commented
8 months ago FWIW I'm building on another machine now (older Ubuntu 22.04.3 LTS) and it seems to work well. I didn't have a lot of the issues I encountered so far (unwind, wasi, etc). Assuming it does complete, can I copy that build over to the machine and continue there? That'd mean building Gecko on this machine then import back with Android Studio on the other machine to build for Lynx.
PS: I'll try to rebuild it all on the same machine obviously, as this is not convenient, so mostly asking due to impatience and curiosity.
PPS: "12:49.60 Your build was successful!" it didn't even take very long.
Utopiah
commented
8 months ago it's really difficult to get reproducible builds or even normal builds but from previous versions with different library dependencies.
Indeed, I might for the sake of the exercise (and my sanity) try this again through a Docker container, e.g https://github.com/Deadolus/android-studio-docker , in order to have something that doesn't depend on small details of my normal working environment.
Utopiah
commented
8 months ago FWIW after ./mach build on the latest Ubuntu I get
Paste the lines between the chevrons (>>> and <<<) into
/home/fabien/Prototypes/firefox-121.0.1/mozconfig:
>>>
# Build GeckoView/Firefox for Android:
ac_add_options --enable-project=mobile/android
# Targeting the following architecture.
# For regular phones, no --target is needed.
# For x86 emulators (and x86 devices, which are uncommon):
# ac_add_options --target=i686
# For newer phones or Apple silicon
# ac_add_options --target=aarch64
# For x86_64 emulators (and x86_64 devices, which are even less common):
# ac_add_options --target=x86_64
<<<
Your system should be ready to build GeckoView/Firefox for Android!
fabien@fabien-CORSAIR-ONE-i160:~/Prototypes/firefox-121.0.1$ ./mach build
whereas on Ubuntu LTS (where I successfully built Gecko then Wolvic) I remember a line that suggested to restart the environment, e.g entering bash again. This is missing here so something is different and maybe causing the build to fail.
Utopiah
commented
8 months ago I'll try to build within a 22.04 container https://hub.docker.com/_/ubuntu/ hopefully that will solve it.
Utopiah
commented
8 months ago Indeed within a container I get :
To build GeckoView/Firefox for Android, please restart the shell (Start a new terminal window)as the last line.
Utopiah
commented
8 months ago Seems to be building. If it succeed I'll share the Dockerfile back.
Utopiah
commented
8 months ago So I'm managing to build Gecko for Android headlessly but the gradle part headlessly remains tricky. I managed to have some of the Android SDK installed, accept certificates, but starting gradlew there are some issues.
While searching in this very repository for details ongradle (to build yet avoiding Android Studio) I found https://github.com/Igalia/wolvic/blob/main/tools/docker/Dockerfile with plenty of interesting solutions. It does seem quite out of date though and untouched for 2 years. Wondering if @svillar knowns if it could be updated.
PS: on the "inspiration" side for the Android SDK https://github.com/mindrunner/docker-android-sdk/blob/main/ubuntu/standalone/Dockerfile seems up to date so maybe using runmymind/docker-android-sdk:ubuntu-standalone as base image could help.
Unfortunately it doesn't seem to play well with mach :
34.66 Looks like you have an outdated Android SDK installed at:
34.66 /opt/android-sdk-linuxFWIW this is what I have so far, before trying again with the docker-android-sdk Ubuntu based image :
FROM ubuntu:22.04
# motivated by https://github.com/Igalia/wolvic/wiki/Developer-workflow#building-gecko
# necessary due to https://github.com/Igalia/wolvic/issues/1270
RUN apt update && apt install -y curl git patch python3 python3-pip xz-utils wget openjdk-17-jre-headless
RUN mkdir Prototypes
WORKDIR Prototypes
ARG VERSION=121.0.1
RUN curl -O https://ftp.mozilla.org/pub/firefox/releases/$VERSION/source/firefox-$VERSION.source.tar.xz
RUN tar -xf firefox-$VERSION.source.tar.xz
RUN git clone https://github.com/Igalia/wolvic-gecko-patches.git
WORKDIR firefox-$VERSION
RUN find ../wolvic-gecko-patches/gecko-$VERSION/ -type f -name "*patch" -print0 | sort -z | xargs -t -n1 -0 patch -p1 -i
RUN echo "ac_add_options --enable-project=mobile/android" > mozconfig
RUN echo "ac_add_options --target=aarch64 --enable-linker=lld" >> mozconfig
RUN ./mach --no-interactive bootstrap --application-choice="GeckoView/Firefox for Android"
RUN bash
# needed to reload some environment
RUN ./mach build
RUN git clone https://github.com/Igalia/wolvic
WORKDIR wolvic
RUN git submodule update --init --recursive
RUN mkdir third_party
WORKDIR third_party
RUN git clone https://github.com/KhronosGroup/OpenXR-SDK
RUN mkdir lynx
WORKDIR lynx
RUN wget https://portal.lynx-r.com/downloads/item/9/download/latest -O loader-release.aar
WORKDIR ../../FWIW appending
RUN cd /opt && wget -q https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip -O android-sdk.zip
RUN cd /opt && unzip android-sdk.zip
RUN cd /opt && rm -f android-sdk.zip
RUN cd /opt && mv cmdline-tools tools
# should probably be 34.sth?
ENV ANDROID_BUILD_TOOLS_VERSION_29 "29.0.2"
ENV ANDROID_SDK_HOME /opt
ENV ANDROID_HOME /opt
ENV PATH ${PATH}:${ANDROID_SDK_HOME}/tools/bin:${ANDROID_SDK_HOME}/platform-tools:/opt/tools:${ANDROID_SDK_HOME}/build-tools/${ANDROID_BUILD_TOOLS_VERSION_29}
# maybe the commandlinetools changed since the version used then, maybe some tools were even removed
# /opt/cmdline-tools/bin/
# apkanalyzer avdmanager lint profgen resourceshrinker retrace screenshot2 sdkmanager
RUN echo y | sdkmanager "ndk-bundle" --sdk_root=$ANDROID_SDK_HOME
RUN echo y | sdkmanager "cmake;3.10.2.4988404" --sdk_root=$ANDROID_SDK_HOME
RUN echo y | sdkmanager "platforms;android-29" --sdk_root=$ANDROID_SDK_HOME
RUN echo y | sdkmanager "build-tools;${ANDROID_BUILD_TOOLS_VERSION_29}" --sdk_root=$ANDROID_SDK_HOMEeven though some wrong versions this still allows gradlew to use the SDK and pull the right versions, e.g CMake 3.22.1, Android SDK Build-Tools 34 v.34.0.0, etc.
Utopiah
commented
8 months ago Ok seems the whole Dockerfile now could look like :
FROM ubuntu:22.04
# motivated by https://github.com/Igalia/wolvic/wiki/Developer-workflow#building-gecko
# necessary due to https://github.com/Igalia/wolvic/issues/1270
RUN apt update && apt install -y curl git patch python3 python3-pip xz-utils wget openjdk-17-jre-headless
# should use a volume here
RUN mkdir Prototypes
WORKDIR Prototypes
# should have used WORKDIR instead... pointless and would take time, space, bandwith to rebuild from this stage/layer
ARG VERSION=121.0.1
RUN curl -O https://ftp.mozilla.org/pub/firefox/releases/$VERSION/source/firefox-$VERSION.source.tar.xz
RUN tar -xf firefox-$VERSION.source.tar.xz
RUN git clone https://github.com/Igalia/wolvic-gecko-patches.git
WORKDIR firefox-$VERSION
RUN find ../wolvic-gecko-patches/gecko-$VERSION/ -type f -name "*patch" -print0 | sort -z | xargs -t -n1 -0 patch -p1 -i
RUN echo "ac_add_options --enable-project=mobile/android" > mozconfig
RUN echo "ac_add_options --target=aarch64 --enable-linker=lld" >> mozconfig
RUN ./mach --no-interactive bootstrap --application-choice="GeckoView/Firefox for Android"
RUN bash
# needed to reload some environment
RUN ./mach build
RUN git clone https://github.com/Igalia/wolvic
WORKDIR wolvic
RUN git submodule update --init --recursive
RUN mkdir third_party
WORKDIR third_party
RUN git clone https://github.com/KhronosGroup/OpenXR-SDK
RUN mkdir lynx
WORKDIR lynx
RUN wget https://portal.lynx-r.com/downloads/item/9/download/latest -O loader-release.aar
WORKDIR ../../
RUN cd /opt && wget -q https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip -O android-sdk.zip
RUN cd /opt && unzip android-sdk.zip
RUN cd /opt && rm -f android-sdk.zip
RUN cd /opt && mv cmdline-tools tools
# maybe the commandlinetools changed since the version used then, maybe some tools were even removed
ENV ANDROID_BUILD_TOOLS_VERSION_34 "34.0.0"
ENV ANDROID_SDK_HOME /opt
ENV ANDROID_HOME /opt
ENV PATH ${PATH}:${ANDROID_SDK_HOME}/tools/bin:${ANDROID_SDK_HOME}/platform-tools:/opt/tools:${ANDROID_SDK_HOME}/build-tools/${ANDROID_BUILD_TOOLS_VERSION_34}
RUN echo y | sdkmanager "ndk-bundle" --sdk_root=$ANDROID_SDK_HOME
RUN echo y | sdkmanager "cmake;3.22.1" --sdk_root=$ANDROID_SDK_HOME
RUN echo y | sdkmanager "platforms;android-34" --sdk_root=$ANDROID_SDK_HOME
RUN echo y | sdkmanager "build-tools;${ANDROID_BUILD_TOOLS_VERSION_34}" --sdk_root=$ANDROID_SDK_HOME
RUN echo /Prototypes/firefox-121.0.1 > local.properties
RUN echo /Prototypes/firefox-121.0.1/obj-aarch64-unknown-linux-android >> local.properties
RUN ./gradlew --no-daemon compileLynxArm64GeckoGenericReleaseSources
RUN ./gradlew --no-daemon assembleLynxArm64GeckoGeneric
RUN ./gradlew --no-daemon bundleLynxArm64GeckoGenericI don't know if it works yet on the device the output as Wolvic-lynx-arm64-gecko-generic-release-unsigned.apk specifically /Prototypes/firefox-121.0.1/wolvic/app/build/outputs/apk/lynxArm64GeckoGeneric/release/Wolvic-lynx-arm64-gecko-generic-release-unsigned.apk
Warning : this is a BIG build virtual 50.6GB and take ~35min on a good desktop.
coderofsalvation
commented
8 months ago great work! if we shoehorn this into a github action then anybody who forks can build it?
Utopiah
commented
8 months ago Indeed but it would need a large runner https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners#machine-sizes-for-larger-runners due to the size.
I managed to build Wolvic 1.7 for Lynx today. Works on device, no problem.
Unfortunately it doesn't support WebXR so I'm trying to do that again but building Gecko first.
I'm following https://github.com/Igalia/wolvic/wiki/Developer-workflow#4-build-gecko and have no problem so far.
Unfortunately with
./mach buildI'm hitting :When I see
__MINGW32__I think of Windows but I'm building on Linux and targeting Android so not sure if relevant.Also I see
ld.lld: error: unable to find library -lunwindyet on theUbuntu 23.10I'm using to build I do haveso any pointer would be welcomed!