search

melis-wallet / melis-cm-client

MIT License
1 stars 1 forks source link

Build failures, how to build APK inside linux container? #5

Closed emanuelb closed 3 years ago

emanuelb commented 3 years ago

Tried to build the 1.6.15 branch and check if the build is reproducible (compare to app downloaded from google-play) & if not what the diff are.

compile with podman build --rm -t melis_ubuntu_build_apk -f Containerfile

Containerfile content is: 

FROM ubuntu:rolling

RUN set -ex; \
    mkdir -p /usr/share/man/man1/; \
    apt-get update; \
    apt-get install --yes --no-install-recommends \
        yarnpkg \
        npm \
        git \
        g++ \
        make \
        bash \
        wget \
        unzip \
        bzip2 \
        patch \
        python3 \
        zipalign \
        openjdk-8-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser; \
    ln -s /usr/bin/yarnpkg /usr/bin/yarn;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="development"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/melis/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/melis/; \
    wget https://github.com/melis-wallet/melis-cm-client/releases/download/1.6.18/ember-leaf-theme-basic-0.40.0.zip; \
    unzip ember-leaf-theme-basic-0.40.0.zip; \
    rm ember-leaf-theme-basic-0.40.0.zip; \
    wget https://github.com/melis-wallet/melis-cm-client/archive/refs/heads/release-1.6.15.zip; \
    unzip release-1.6.15.zip; \
    rm release-1.6.15.zip; \
    git clone https://github.com/melis-wallet/melis-cm-client/; \
    cd melis-cm-client; \
    git checkout release-1.6.15;

WORKDIR /home/appuser/app/melis/

Then run:

podman run --rm --name melis_ubuntu_built_apk -ti melis_ubuntu_build_apk

inside container what commands to run in order to compile the app and generate a apk?

WIP will edit later with commands taken from readme and different tries to compile the app which failed with different errors.

Try 1:

cd melis-cm-client
yarn global add bower ember-cli corber;
yarn
/home/appuser/.yarn/bin/bower install;
# MANUAL-POSTINSTALL-STEP -> from postinstall of master (otherwise not found errors of `/node_modules/emblem/dist/cjs/emblem.js`)
cd ./node_modules/emblem;
yarn;
yarn run build;
cd  ../../
# End post-install step, run it again if dist folder deleted.

PATH="$PATH:/home/appuser/.yarn/bin/" ./wallet &
DEPLOY_TARGET='production' /home/appuser/.yarn/bin/ember build --environment production

PATH="$PATH:/home/appuser/.yarn/bin/" build/cordova-setup.sh android production
PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh production 1.6.15 changeme

try 2:

npm install -g yarn bower ember-cli
su appuser
cd melis-cm-client
yarn
bower install
cd ./node_modules/emblem;
yarn;
yarn run build;
cd  ../../

ember serve &
./wallet &

build/cordova-setup.sh android production
DEPLOY_TARGET='production' ember cordova:build --environment=production --platform=android --release

errors list:

error when running build/cordova-setup.sh

build/cordova-setup.sh: line 85: corber: command not found

fixed by install it globally, but then other errors...

emanuelb commented 3 years ago

Building tag 1.6.21 also fail with errors:

  1. Cannot find module 'corber/lib/targets/cordova/utils/cordova-assets' , installed cordova-assets to fix.
  2. warning: (when running ember build) [TerserWriter][WARN] (broccoli-terser-sourcemap) "ember.map" referenced in "assets/vendor.js" could not be found
  3. CordovaErrors: when commands run /home/appuser/.yarn/bin/corber plugin add phonegap-plugin-mobile-accessibility 
    DEPRECATION: ember-i18n has been deprecated in favor of ember-intl
attempting to add plugin 'phonegap-plugin-mobile-accessibility'...
Failed to install 'phonegap-plugin-mobile-accessibility': CordovaError: Failed to fetch plugin https://github.com/apache/cordova-plugin-device.git via registry.

ContainerFile:

FROM ubuntu:rolling

RUN set -ex; \
    mkdir -p /usr/share/man/man1/; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
        yarnpkg \
        npm \
        git \
        g++ \
        make \
        bash \
        wget \
        unzip \
        bzip2 \
        patch \
        python3 \
        zipalign \
        watchman \
#       phantomjs \
        openjdk-8-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser; \
    ln -s /usr/bin/yarnpkg /usr/bin/yarn;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="development"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/tools/" "/home/appuser/app/melis/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/tools/; \
    wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    tar -xvjf phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    echo "86dd9a4bf4aee45f1a84c9f61cf1947c1d6dce9b9e8d2a907105da7852460d2f  phantomjs-2.1.1-linux-x86_64.tar.bz2" | sha256sum -c; \ 
    rm phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    cd /home/appuser/app/melis/; \
    wget https://github.com/melis-wallet/melis-cm-client/releases/download/1.6.21/ember-leaf-theme-basic-0.40.0.zip; \
    unzip ember-leaf-theme-basic-0.40.0.zip; \
    echo "e8ee479154746ccba638be368fb49f328acbb10166f60c56988b1fb2df3375c6  ember-leaf-theme-basic-0.40.0.zip" | sha256sum -c; \ 
    rm ember-leaf-theme-basic-0.40.0.zip; \
    wget https://github.com/melis-wallet/melis-cm-client/archive/refs/tags/1.6.21.zip; \
    unzip 1.6.21.zip; \
    rm 1.6.21.zip; \
    git clone https://github.com/melis-wallet/melis-cm-client/; \
    cd melis-cm-client; \
    git checkout 1.6.21;

WORKDIR /home/appuser/app/melis/

build with podman build --pull --rm -t melis_lv_build_apk -f ContainerFile run with podman run --rm --name melis_lv_built_apk -ti melis_lv_build_apk

inside it run the below commands: (if output stuck push "Enter")

yarn global add bower ember-cli corber;
PATH="$PATH:/home/appuser/app/tools/phantomjs-2.1.1-linux-x86_64/bin" yarn global add cordova-assets
cd melis-cm-client-1.6.21
yarn
/home/appuser/.yarn/bin/bower install;
yarn postinstall
PATH="$PATH:/home/appuser/.yarn/bin/" ./wallet &
DEPLOY_TARGET='production' /home/appuser/.yarn/bin/ember build --environment production --platform=android
PATH="$PATH:/home/appuser/.yarn/bin/" build/cordova-setup.sh android production
# select android for init in select dialog
yarn remove corber ember-cli-sri
PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh production 1.6.21 changeme
ghost commented 3 years ago

Can you please try the current release branch following the instructions in the README for "Building a production application" ?

ghost commented 3 years ago

A side note: the corber init stage should not ask if you want to init an android application, but I have reports that it is happening in some cases. I am looking into it. If it happens (ie: if at some point you're asked to initialize the project for "none", "android", and possibly "ios"), pick "none"

ghost commented 3 years ago

@emanuelb: i am trying to reproduce your build using your container and the current public release branch.

yarn global add bower ember-cli corber cordova
PATH="$PATH:/home/appuser/app/tools/phantomjs-2.1.1-linux-x86_64/bin" yarn global add cordova-assets
cd melis-cm-client
yarn
/home/appuser/.yarn/bin/bower install;
yarn postinstall
PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh regtest

Build goes straight on successfully until the cordova application is built:

(node:3822) UnhandledPromiseRejectionWarning: CordovaError: Could not find an installed version of Gradle either in Android Studio,
or on your system to install the gradle wrapper. Please include gradle 
in your path, or install Android Studio

which is to be expected as I don't see the Android SDK being installed anywhere.

See: https://gist.github.com/lele-melis/033e14ccb4e9c0be1405aa82039a44d0

emanuelb commented 3 years ago

gradle can be installed by:

DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends gradle

also calling ./gradlew (if it exists/already created, install the gradle & android SDK), thus by default I don't install them into the image as the build-process may install them themself.

Android SDK can be installed manually like:

cd /home/appuser/app/sdk/; \
wget https://dl.google.com/android/repository/commandlinetools-linux-7302050_latest.zip; \
echo "7a00faadc0864f78edd8f4908a629a46d622375cbe2e5814e82934aebecdb622  commandlinetools-linux-7302050_latest.zip" | sha256sum -c; \
unzip commandlinetools-linux-7302050_latest.zip; \
rm commandlinetools-linux-7302050_latest.zip; \
/home/appuser/app/sdk/cmdline-tools/bin/sdkmanager --sdk_root=/home/appuser/app/sdk/ --install "build-tools;30.0.3";

@lele-melis The problem is elsewhere, there failures in plugins downloads which looks maybe it's because the container use newer versions for some manually installed deps (ember-cli / corber) if you can tell which is the version of this deps when compiling is work the container can install this specific versions. for example another bitcoin wallet monya declare in README: https://github.com/monya-wallet/monya/#cordova-build npm install -g cordova@7.1.0 Caution: Cordova version must be 7.1. 8.0 is not supported. And indeed build failed when default version of cordova was installed, I guess the same happen here with either ember-cli or corber or both deps.

Here Containerfile that install both gradle & android-SDK & downloading latest commit only:

FROM ubuntu:rolling

RUN set -ex; \
    mkdir -p /usr/share/man/man1/; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
        yarnpkg \
        npm \
        git \
        g++ \
        make \
        bash \
        wget \
        unzip \
        bzip2 \
        patch \
        gradle \
        python3 \
        zipalign \
        watchman \
#       phantomjs \
        openjdk-8-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser; \
    ln -s /usr/bin/yarnpkg /usr/bin/yarn;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="development"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/tools/" "/home/appuser/app/melis/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/sdk/; \
    wget https://dl.google.com/android/repository/commandlinetools-linux-7302050_latest.zip; \
    echo "7a00faadc0864f78edd8f4908a629a46d622375cbe2e5814e82934aebecdb622  commandlinetools-linux-7302050_latest.zip" | sha256sum -c; \
    unzip commandlinetools-linux-7302050_latest.zip; \
    rm commandlinetools-linux-7302050_latest.zip; \
    /home/appuser/app/sdk/cmdline-tools/bin/sdkmanager --sdk_root=/home/appuser/app/sdk/ --install "build-tools;30.0.3"; \    
    cd /home/appuser/app/tools/; \
    wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    tar -xvjf phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    echo "86dd9a4bf4aee45f1a84c9f61cf1947c1d6dce9b9e8d2a907105da7852460d2f  phantomjs-2.1.1-linux-x86_64.tar.bz2" | sha256sum -c; \ 
    rm phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    cd /home/appuser/app/melis/; \
    wget https://github.com/melis-wallet/melis-cm-client/releases/download/1.6.21/ember-leaf-theme-basic-0.40.0.zip; \
    unzip ember-leaf-theme-basic-0.40.0.zip; \
    echo "e8ee479154746ccba638be368fb49f328acbb10166f60c56988b1fb2df3375c6  ember-leaf-theme-basic-0.40.0.zip" | sha256sum -c; \ 
    rm ember-leaf-theme-basic-0.40.0.zip; \
    git clone --depth 1 https://github.com/melis-wallet/melis-cm-client/;

WORKDIR /home/appuser/app/melis/melis-cm-client/

build with podman build --pull --rm -t melis_lv_build_apk -f ContainerFileN run with podman run --rm --name melis_lv_built_apk -ti melis_lv_build_apk

command in:

cd /home/appuser/app/melis;
yarn global add bower ember-cli corber;
PATH="$PATH:/home/appuser/app/tools/phantomjs-2.1.1-linux-x86_64/bin" yarn global add cordova-assets
cd melis-cm-client
yarn
/home/appuser/.yarn/bin/bower install;
yarn postinstall

Then run:

PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh production 1.6.21 changeme

or as before:

PATH="$PATH:/home/appuser/.yarn/bin/" ./wallet &
DEPLOY_TARGET='production' /home/appuser/.yarn/bin/ember build --environment production --platform=android
PATH="$PATH:/home/appuser/.yarn/bin/" build/cordova-setup.sh android production
# select android for init in select dialog
yarn remove corber ember-cli-sri
PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh production 1.6.21 changeme

Same errors as before:

(node:1206) UnhandledPromiseRejectionWarning: CordovaError: Failed to fetch plugin cordova-plugin-qrscanner via registry.

Also error:

Unrecognized option: --illegal-access=permit Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. (node:1822) UnhandledPromiseRejectionWarning: Error: Command failed with exit code 1: /home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/gradlew cdvBuildRelease -b /home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/build.gradle at makeError (/home/appuser/app/melis/melis-cm-client/corber/cordova/node_modules/execa/lib/error.js:59:11) at handlePromise (/home/appuser/app/melis/melis-cm-client/corber/cordova/node_modules/execa/index.js:114:26) at processTicksAndRejections (internal/process/task_queues.js:97:5) (node:1822) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:1822) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. Linting corber/cordova/www/index.html...

Running manually the above command that failed again worked:

/home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/gradlew cdvBuildRelease -b /home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/build.gradle

It does install Android 29 SDK, and even generate APK (but many errors in the process)

Running diff --recursive --brief to compare against: https://github.com/melis-wallet/melis-cm-client/releases/download/1.6.21/melis-production.apk

As there is files/folders only in "fromGithub" (like plugins & raw & layout) it's looks like app compilation is not complete (APK generated is broken)

Files /localBuild/AndroidManifest.xml and /fromGithub/AndroidManifest.xml differ
Files /localBuild/META-INF/MANIFEST.MF and /fromGithub/META-INF/MANIFEST.MF differ
Only in /fromGithub/META-INF: MELIS_AN.RSA
Only in /fromGithub/META-INF: MELIS_AN.SF
Only in /fromGithub/assets/www/assets/icons: appicon-192-6f484825d4b00c4452786124874d4f96.png
Only in /localBuild/assets/www/assets/icons: appicon-192-788bc2cef6d6532f3f62d5cae2067c8a.png
Only in /fromGithub/assets/www/assets/icons: appicon-512-300aa51fa82dabba7958eb8c0b74930e.png
Only in /localBuild/assets/www/assets/icons: appicon-512-4a6d86c9fcaecc2ea3ac6171fb10d132.png
Only in /fromGithub/assets/www/assets: melis-cm-client-7e8a84178a1b48d65ca500b1ebad30df.js
Only in /fromGithub/assets/www/assets: melis-cm-client-8a678e401f3eb02b8a4c60c9b3fbaf36.map
Only in /localBuild/assets/www/assets: melis-cm-client-8c7e463f93a33904a50af54456574bee.js
Only in /localBuild/assets/www/assets: melis-cm-client-d664eb1e2340acd8e313c52337b965f4.map
Only in /fromGithub/assets/www/assets: vendor-255049e484d0fb283a02e6012eb13e31.js
Only in /fromGithub/assets/www/assets: vendor-37416b8c07713f4b1267ba43cfd8aef6.css
Only in /localBuild/assets/www/assets: vendor-3d2eb2ab04d521cd09654b23f1e95219.map
Only in /localBuild/assets/www/assets: vendor-89b88b399bca09ad50aa1aa23e71f91d.css
Only in /localBuild/assets/www/assets: vendor-8d9b1707be2647a9ab93da5da213db00.js
Only in /fromGithub/assets/www/assets: vendor-d588b3a9c5c2d0ce730c73e2fa76b5e9.map
Files /localBuild/assets/www/cordova_plugins.js and /fromGithub/assets/www/cordova_plugins.js differ
Files /localBuild/assets/www/index.html and /fromGithub/assets/www/index.html differ
Files /localBuild/assets/www/manifest.webmanifest and /fromGithub/assets/www/manifest.webmanifest differ
Only in /fromGithub/assets/www: plugins
Files /localBuild/assets/www/sw.js and /fromGithub/assets/www/sw.js differ
Files /localBuild/classes.dex and /fromGithub/classes.dex differ
Only in /fromGithub/res: anim
Only in /fromGithub/res: color
Only in /fromGithub/res: color-v11
Only in /fromGithub/res: color-v23
Only in /fromGithub/res: drawable
Only in /fromGithub/res: drawable-hdpi-v4
Only in /fromGithub/res: drawable-ldrtl-hdpi-v4
Only in /fromGithub/res: drawable-ldrtl-mdpi-v4
Only in /fromGithub/res: drawable-ldrtl-xhdpi-v4
Only in /fromGithub/res: drawable-ldrtl-xxhdpi-v4
Only in /fromGithub/res: drawable-ldrtl-xxxhdpi-v4
Only in /fromGithub/res: drawable-mdpi-v4
Only in /fromGithub/res: drawable-nodpi-v4
Only in /fromGithub/res: drawable-v21
Only in /fromGithub/res: drawable-v23
Only in /fromGithub/res: drawable-xhdpi-v4
Only in /fromGithub/res: drawable-xxhdpi-v4
Only in /fromGithub/res: drawable-xxxhdpi-v4
Only in /fromGithub/res: layout
Only in /fromGithub/res: raw
Files /localBuild/res/xml/config.xml and /fromGithub/res/xml/config.xml differ
Files /localBuild/resources.arsc and /fromGithub/resources.arsc differ
ghost commented 3 years ago

@emanuelb: can you please do the following, as stated previously:

emanuelb commented 3 years ago

@lele-melis

please use the head for release branch and not the 1.6.21 package

already did, the code in latest Containerfile in previous comment does: git clone --depth 1 https://github.com/melis-wallet/melis-cm-client/; (clone latest commit in repo in release branch, which is 8037d78bb35edf2dafa260e00d53673c84a1bf21) it just download the ember-leaf-theme-basic-0.40.0.zip file from 1.6.21 release, as it's the latest version available to download.

follow the instructions in the README for the release branch, they are very different from what you're doing.

Can you write the instructions here? better as in step-by-step or list of commands to run? it looks the same to me... to just run the command: PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh production 1.6.15 changeme which I did and it failed as well.

answer "none" when you are asked if you want to initialize corber with android.

Yes, already did that.

Will be good if you can rewrite the above Containerfile to compile the app, to make it work... The next error is fixed probably by using specific versions for ember-cli corber packages, what version do you use for this packages? looks like older version is needed...

ghost commented 3 years ago

I am using this Containerfile 

FROM ubuntu:20.10

RUN set -ex; \
    mkdir -p /usr/share/man/man1/; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
        yarnpkg \
        npm \
                vim \
        git \
        g++ \
        make \
        bash \
        wget \
        unzip \
        bzip2 \
        patch \
        python3 \
        zipalign \
        watchman \
#       phantomjs \
                gradle \
                ssh \
        openjdk-8-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser; \
    ln -s /usr/bin/yarnpkg /usr/bin/yarn;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="development"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/tools/" "/home/appuser/app/melis/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/sdk/; \
    wget https://dl.google.com/android/repository/commandlinetools-linux-7302050_latest.zip; \
    echo "7a00faadc0864f78edd8f4908a629a46d622375cbe2e5814e82934aebecdb622  commandlinetools-linux-7302050_latest.zip" | sha256sum -c; \
    unzip commandlinetools-linux-7302050_latest.zip; \
    rm commandlinetools-linux-7302050_latest.zip; \
    /home/appuser/app/sdk/cmdline-tools/bin/sdkmanager --sdk_root=/home/appuser/app/sdk/ --install "build-tools;30.0.3"; \    
    cd /home/appuser/app/tools/; \
    wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    tar -xvjf phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    echo "86dd9a4bf4aee45f1a84c9f61cf1947c1d6dce9b9e8d2a907105da7852460d2f  phantomjs-2.1.1-linux-x86_64.tar.bz2" | sha256sum -c; \ 
    rm phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    cd /home/appuser/app/melis/; \
    wget https://github.com/melis-wallet/melis-cm-client/releases/download/1.6.21/ember-leaf-theme-basic-0.40.0.zip; \
    unzip ember-leaf-theme-basic-0.40.0.zip; \
    echo "e8ee479154746ccba638be368fb49f328acbb10166f60c56988b1fb2df3375c6  ember-leaf-theme-basic-0.40.0.zip" | sha256sum -c; \ 
    rm ember-leaf-theme-basic-0.40.0.zip; #\
    git clone https://github.com/melis-wallet/melis-cm-client/; \
    cd melis-cm-client;

WORKDIR /home/appuser/app/melis/

Building like this:

cd /home/appuser/app/melis;
yarn global add bower ember-cli corber cordova
PATH="$PATH:/home/appuser/app/tools/phantomjs-2.1.1-linux-x86_64/bin" yarn global add cordova-assets
cd melis-cm-client
yarn
/home/appuser/.yarn/bin/bower install;
yarn postinstall
PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh production

I can successfully build a production *.apk

appuser@74be3bbdbe88:~/app/melis/melis-cm-client$ cd build/dist/
appuser@74be3bbdbe88:~/app/melis/melis-cm-client/build/dist$ ll
total 8124
drwxr-xr-x 2 appuser appuser    4096 May 24 12:44 ./
drwxr-xr-x 4 appuser appuser    4096 May 24 12:36 ../
-rw-r--r-- 1 appuser appuser       0 May 24 12:36 .gitkeep
-rw-r--r-- 1 appuser appuser 8309382 May 24 12:44 melis-production-unsigned.apk

https://gist.github.com/lele-melis/8a12e14071cf572e8764ff327dcaae8c

There are a few errors, indeed, but the resulting apk is complete:

appuser@74be3bbdbe88:~/app/melis/melis-cm-client/build/dist$ diff --recursive --brief local/  production/
Files local/META-INF/MANIFEST.MF and production/META-INF/MANIFEST.MF differ
Only in production/META-INF: MELIS_AN.RSA
Only in production/META-INF: MELIS_AN.SF
Only in production/assets/www/assets/icons: appicon-192-6f484825d4b00c4452786124874d4f96.png
Only in local/assets/www/assets/icons: appicon-192-788bc2cef6d6532f3f62d5cae2067c8a.png
Only in production/assets/www/assets/icons: appicon-512-300aa51fa82dabba7958eb8c0b74930e.png
Only in local/assets/www/assets/icons: appicon-512-4a6d86c9fcaecc2ea3ac6171fb10d132.png
Only in production/assets/www/assets: vendor-37416b8c07713f4b1267ba43cfd8aef6.css
Only in local/assets/www/assets: vendor-3d2eb2ab04d521cd09654b23f1e95219.map
Only in local/assets/www/assets: vendor-8d9b1707be2647a9ab93da5da213db00.js
Only in production/assets/www/assets: vendor-a8c2e3420beea503e97bb2b4e9b363f3.map
Only in local/assets/www/assets: vendor-b3c5f6d9be246e70c6b7e5f80648f19f.css
Only in production/assets/www/assets: vendor-b7c9ff16e73af3132bf891b8195fc9b0.js
Files local/assets/www/index.html and production/assets/www/index.html differ
Files local/assets/www/manifest.webmanifest and production/assets/www/manifest.webmanifest differ
Files local/assets/www/sw.js and production/assets/www/sw.js differ

(These diffs are between a build of the release head in "your" container local vs our production pipeline production ) There are still a few differences, mainly three:

CACHE_BUSTER="ffd8c7cc0bd149e507583426a1e71a5a38db98bb

All the remaining (index.html, manifest.webmanifest, *.map) files only differ because they reference one of the fingerprint-named files.

ghost commented 3 years ago

This, of course, is not a reproducible build yet, but if and when we can overcome the vendor.* issue above, we can tag a version that will.

emanuelb commented 3 years ago

All the diffs between the local & production builds should be fixed, which is related to issue about reproducible-builds at #1 Regarding file ordering issue, a workaround that might help is using disorderfs tool: https://salsa.debian.org/reproducible-builds/disorderfs see for example as I did in: https://gitlab.com/walletscrutiny/walletScrutinyCom/-/issues/197#note_543234399 Anyway using disorderfs is kinda "wrong" and the fix should be applied elsewhere (to do the sorting)

Regarding randomness see: https://reproducible-builds.org/docs/randomness/ & SOURCE_DATE_EPOCH https://reproducible-builds.org/docs/source-date-epoch/ (using some data as pre-seed for PRNG is possible fix to reproducible randomness)

Also you can use diffoscope tool to compare the APKs https://diffoscope.org (which is widely used by reproducible-builds efforts)

Why is ssh dep is installed in your Containerfile? Adding the openssh-client dep is probably better (if no need for server-side which is openssh-server openssh-sftp-server deps of ssh) but it still looks kinda "wrong" if the build process require ssh client...

Also it still ask Which platforms should we init with? -> none

using ubuntu:21.04 which is latest version (The 20.10 is old/obsolete) instead of 20.10 without ssh+vim deps have the same result as before, no melis-production-unsigned.apk file is generated, errors are (and also issue #12) : 

(node:279) UnhandledPromiseRejectionWarning: CordovaError: Failed to fetch platform cordova-android@~9.0.0
Probably this is either a connection problem, or platform spec is incorrect.
Check your connection and platform name/version/URL.

DEPRECATION: ember-i18n has been deprecated in favor of ember-intl
attempting to add plugin 'ionic-plugin-keyboard'...
(node:586) UnhandledPromiseRejectionWarning: CordovaError: Failed to fetch plugin ionic-plugin-keyboard via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.

Adding cordova.js to corber/cordova/www/index.html...

Unrecognized option: --illegal-access=permit
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
(node:1156) UnhandledPromiseRejectionWarning: Error: Command failed with exit code 1: /home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/gradlew cdvBuildRelease -b /home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/build.gradle
    at makeError (/home/appuser/app/melis/melis-cm-client/corber/cordova/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/home/appuser/app/melis/melis-cm-client/corber/cordova/node_modules/execa/index.js:114:26)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:1156) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:1156) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Linting corber/cordova/www/index.html...

WARNING: corber 

corber/cordova/www/index.html
There are remaining paths beginning with / in your code. They likely will not work.  Line 16  href-attribute contains unsupported path relative to root: /assets/icons/appicon-32-13273f449257c4d06f6cb9c2e6f7cd6e.png
  Line 44  src-attribute contains unsupported path relative to root: /sw-registration-2593205ffca1072359dd40798ae2fbd3.js

✖ 2 problem(s)
corber project built
Artifact will not be signed
cp: cannot stat '/home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/app/build/outputs/apk//release/app-release-unsigned.apk': No such file or directory
*** BUILD FINISHED

Containerfile used:

FROM ubuntu:21.04

RUN set -ex; \
    mkdir -p /usr/share/man/man1/; \
    apt-get update; \
    DEBIAN_FRONTEND=noninteractive apt-get install --yes -o APT::Install-Suggests=false --no-install-recommends \
        yarnpkg \
        npm \
        git \
        g++ \
        make \
        bash \
        wget \
        unzip \
        bzip2 \
        patch \
        gradle \
        python3 \
        zipalign \
        watchman \
#       phantomjs \
#       openssh-client  \
        openjdk-8-jdk ; \
    rm -rf /var/lib/apt/lists/*; \
    useradd -ms /bin/bash appuser; \
    ln -s /usr/bin/yarnpkg /usr/bin/yarn;

USER appuser

ENV ANDROID_SDK_ROOT="/home/appuser/app/sdk" \
    ANDROID_HOME="/home/appuser/app/sdk" \
    NODE_ENV="development"

RUN set -ex; \
    mkdir -p "/home/appuser/app/sdk/licenses" "/home/appuser/app/tools/" "/home/appuser/app/melis/"; \
    printf "\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > "/home/appuser/app/sdk/licenses/android-sdk-license"; \
    cd /home/appuser/app/sdk/; \
    wget https://dl.google.com/android/repository/commandlinetools-linux-7302050_latest.zip; \
    echo "7a00faadc0864f78edd8f4908a629a46d622375cbe2e5814e82934aebecdb622  commandlinetools-linux-7302050_latest.zip" | sha256sum -c; \
    unzip commandlinetools-linux-7302050_latest.zip; \
    rm commandlinetools-linux-7302050_latest.zip; \
    /home/appuser/app/sdk/cmdline-tools/bin/sdkmanager --sdk_root=/home/appuser/app/sdk/ --install "build-tools;30.0.3"; \    
    cd /home/appuser/app/tools/; \
    wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    tar -xvjf phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    echo "86dd9a4bf4aee45f1a84c9f61cf1947c1d6dce9b9e8d2a907105da7852460d2f  phantomjs-2.1.1-linux-x86_64.tar.bz2" | sha256sum -c; \ 
    rm phantomjs-2.1.1-linux-x86_64.tar.bz2; \
    cd /home/appuser/app/melis/; \
    wget https://github.com/melis-wallet/melis-cm-client/releases/download/1.6.21/ember-leaf-theme-basic-0.40.0.zip; \
    unzip ember-leaf-theme-basic-0.40.0.zip; \
    echo "e8ee479154746ccba638be368fb49f328acbb10166f60c56988b1fb2df3375c6  ember-leaf-theme-basic-0.40.0.zip" | sha256sum -c; \ 
    rm ember-leaf-theme-basic-0.40.0.zip; \
    yarn global add bower ember-cli corber cordova; \
    PATH="$PATH:/home/appuser/app/tools/phantomjs-2.1.1-linux-x86_64/bin" yarn global add cordova-assets; \
    git clone --depth 1 https://github.com/melis-wallet/melis-cm-client/;

WORKDIR /home/appuser/app/melis/melis-cm-client/

RUN set -ex; \
    yarn; \
    /home/appuser/.yarn/bin/bower install; \
    yarn postinstall

and in the container run:

PATH="$PATH:/home/appuser/.yarn/bin/" build/build-android.sh production

Using the same steps in your previous comments result in generated APK! (your Containerfile with char # removed & same steps) still there are errors shown:

(node:1258) UnhandledPromiseRejectionWarning: Error: Source path does not exist: res/melis-badger.png
    at updatePathInternal (/home/appuser/app/melis/melis-cm-client/corber/cordova/node_modules/cordova-common/src/FileUpdater.js:126:19)
    at /home/appuser/app/melis/melis-cm-client/corber/cordova/node_modules/cordova-common/src/FileUpdater.js:192:16
    at Array.map (<anonymous>)
    at Object.updatePaths (/home/appuser/app/melis/melis-cm-client/corber/cordova/node_modules/cordova-common/src/FileUpdater.js:190:40)
    at updateIcons (/home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/cordova/lib/prepare.js:439:17)
    at /home/appuser/app/melis/melis-cm-client/corber/cordova/platforms/android/cordova/lib/prepare.js:88:9
    at async Promise.all (index 0)
(node:1258) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:1258) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

attempting to add plugin 'cordova-plugin-fcm-ng'...
(node:2157) UnhandledPromiseRejectionWarning: CordovaError: Variable(s) missing (use: --variable PAGE_LINK_DOMAIN=value).
    at Object.mergeVariables (/home/appuser/.config/yarn/global/node_modules/cordova-lib/src/cordova/plugin/util.js:77:15)
    at /home/appuser/.config/yarn/global/node_modules/cordova-lib/src/cordova/plugin/add.js:102:40
(node:2157) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:2157) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Closing this issue as it's about having Containerfile to compile latest version of the app which is written in previous comment (# char should be removed, some improvments can be made to install less deps, bit it works...)