[Feature]: Consider Reproducible Builds

xrviv commented 2 months ago

Describe the problem

Hi, my name is Danny, and I work with Walletscrutiny.com.

I've been trying to build your app from source according to the instructions at: BUILD.md, but always seem to run into trouble.

My first attempts were to try and build using a Dockerfile to automate the process, but I've managed to reach a block and couldn't get past.

My second attempt was to try building from within a running Docker and have come across the same problem.

My third attempt, I just tried to build locally on my machine and have come across some dependency problems:

Here is a pastebin of my third attempt

Describe the solution

More detailed instructions on how to build from source including ideal build environment
If possible, a Dockerfile to help automate the process, so it would easily be reproduced by others

Additional context

No response

pwltr commented 2 months ago

Hi there, without me trying to reproduce this on a new machine could you try the following:

in case you have Android Studio installed, update to the latest version and open ~/work/builds/bitkit/bitkit/android, let it run gradle sync
run yarn clean
review the react-native environment setup at https://reactnative.dev/docs/environment-setup?os=linux&platform=android
instead of yarn bundle run yarn android to run the development build

xrviv commented 2 months ago

Thank you for the response. I will be attempting this now. Apologies for the late reply, we just completed the builds for other apps.

xrviv commented 2 months ago

So update, the build failed yet again following the new instructions. It would be helpful to have a dockerfile for the builds, Our process usually involves a bash script, and a dockerfile. You can see it in action if you clone https://gitlab.com/walletscrutiny/walletScrutinyCom and look into the https://gitlab.com/walletscrutiny/walletScrutinyCom/scripts/test/android/ folder. There are some templates, I'll provide a sample here for bluewallet:

FROM docker.io/node:18-bullseye-slim

ARG UID=1000
ARG TAG
ARG VERSION

RUN set -ex; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes \
      -o APT::Install-Suggests=false --no-install-recommends \
      patch git openjdk-11-jre-headless openjdk-11-jdk; \
    rm -rf /var/lib/apt/lists/*; \
    deluser node; \
    useradd --uid $UID --create-home --shell /bin/bash appuser; \
    mkdir -p /Users/runner/work/1/; \
    chown -R appuser:appuser /Users/;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/sdk" \
    ANDROID_HOME="/home/appuser/sdk" \
    NODE_ENV="production"

RUN set -ex; \
    mkdir -p "/home/appuser/sdk/licenses"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/sdk/licenses/android-sdk-license"; \
    cd /Users/runner/work/1/; \
    git clone --branch $TAG https://github.com/BlueWallet/BlueWallet /Users/runner/work/1/s/;

WORKDIR /Users/runner/work/1/s/

RUN set -ex; \
    npm config set fetch-retry-maxtimeout 600000; \
    npm config set fetch-retry-mintimeout 100000; \
    npm install --production --no-optional --omit=optional --no-audit --no-fund --ignore-scripts; \
    npm run postinstall; \
    # Work around issue with realm: https://github.com/realm/realm-js/issues/6204#issuecomment-1772638401
    rm -rf node_modules/realm; npm install realm; \
    echo '"master"' > current-branch.json;

RUN set -ex; \
    cd /Users/runner/work/1/s/android; \
    ./gradlew assembleRelease \
        -Dorg.gradle.internal.http.socketTimeout=600000 \
        -Dorg.gradle.internal.http.connectionTimeout=600000

and its corresponding bash script:

#!/bin/bash

repo=https://github.com/BlueWallet/BlueWallet
tag="v$versionName"
builtApk=$workDir/app-release-unsigned.apk

test() {
  podman rmi bluewallet -f
  podman build \
    --tag bluewallet \
    --cgroup-manager cgroupfs \
    --ulimit nofile=16384:16384 \
    --build-arg UID=$(id -u) \
    --build-arg TAG=$tag \
    --build-arg VERSION=$versionCode \
    --file $SCRIPT_DIR/test/android/io.bluewallet.bluewallet.dockerfile
  podman run \
    -it \
    --volume $workDir:/mnt \
    --rm \
    -u root \
    bluewallet \
    bash -c \
      'cp /Users/runner/work/1/s/android/app/build/outputs/apk/release/*.apk /mnt/'

  podman rmi bluewallet -f
  podman image prune -f
}

pwltr commented 2 months ago

It would be helpful to have a dockerfile for the builds, Our process usually involves a bash script, and a dockerfile

I agree it would be a good thing to have but probably not a priority for us right now. Will look into it in the near future, thanks for the pointers.

If you want to describe the build errors you're seeing with manual building I will try to help. Otherwise I'll come back to this soon to try to reproduce from a new setup.

xrviv commented 1 month ago

I was able to run the build in an AVD using yarn android

However, here's the most recent results upon running yarn bundle:

> A failure occurred while executing com.android.build.gradle.tasks.PackageAndroidArtifact$IncrementalSplitterRunnable

   > com.android.ide.common.signing.KeytoolException: Failed to read key androiddebugkey from store "/home/dannybuntu/work/

builds/bitkit-05-07/bitkit/android/app/debug.keystore": Cannot recover key

debug.keystore is in its original location in bitkit/android/app
BITKIT_UPLOAD lines have been added to ~/.gradle/gradle.properties just like the instructions said
./gradlew signingReport does not report any missing keystores.

basantagoswami commented 3 weeks ago

From a conversation on Bitkit's Telegram channel:

Corey Phillips: [link] Bitkit is not going to be reproducible until we release the .env.production file to build the apk with. So I wouldn't waste too much time on that for now. I'm sure there will be additional tweaks we'll need to make to ensure reproducibility beyond that as well.

So we are going to be marking Bitkit as non reproducible, till this issue is resolved.

synonymdev / bitkit