search

DeviceFarmer / stf

Control and manage Android devices from your browser.
https://devicefarmer.github.io
Other
3.47k stars 488 forks source link

armhf docker file does not build #459

Open gounthar opened 2 years ago

gounthar commented 2 years ago

Do you see errors or warnings in the stf local output? If so, please paste them or the full log here.

Does the armhf docker image still works for you in the CI? I tried to build it on an armv7 machine and got this error:


added 2069 packages from 1933 contributors and audited 2090 packages in 2813.53s

30 packages are looking for funding
  run `npm fund` for details

found 206 vulnerabilities (32 low, 73 moderate, 78 high, 23 critical)
  run `npm audit fix` to fix them, or `npm audit` for details
+ echo --- Assembling app
+ echo npm pack
+ su stf -s /bin/bash
--- Assembling app
npm WARN prepublish-on-install As of npm@5, `prepublish` scripts are deprecated.
npm WARN prepublish-on-install Use `prepare` for build steps and `prepublishOnly` for upload-only.
npm WARN prepublish-on-install See the deprecation note in `npm help scripts` for more information.

> @devicefarmer/stf@3.6.1 prepublish /tmp/build
> bower install && not-in-install && gulp build || in-install

{ AssertionError [ERR_ASSERTION]: Task function must be specified
    at Gulp.set [as _setTask] (/tmp/build/node_modules/undertaker/lib/set-task.js:10:3)
    at Gulp.task (/tmp/build/node_modules/undertaker/lib/task.js:13:8)
    at Object.<anonymous> (/tmp/build/gulpfile.js:79:6)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at requireOrImport (/tmp/build/node_modules/gulp/node_modules/gulp-cli/lib/shared/require-or-import.js:19:11)
    at execute (/tmp/build/node_modules/gulp/node_modules/gulp-cli/lib/versioned/^4.0.0/index.js:37:3)
    at Liftoff.handleArguments (/tmp/build/node_modules/gulp/node_modules/gulp-cli/index.js:211:24)
    at Liftoff.execute (/tmp/build/node_modules/liftoff/index.js:201:12)
    at module.exports (/tmp/build/node_modules/flagged-respawn/index.js:51:3)
    at Liftoff.<anonymous> (/tmp/build/node_modules/liftoff/index.js:191:5)
    at /tmp/build/node_modules/liftoff/index.js:149:9
    at /tmp/build/node_modules/v8flags/index.js:145:14
    at /tmp/build/node_modules/graceful-fs/graceful-fs.js:61:14
    at FSReqWrap.oncomplete (fs.js:135:15)
  generatedMessage: false,
  name: 'AssertionError [ERR_ASSERTION]',
  code: 'ERR_ASSERTION',
  actual: false,
  expected: true,
  operator: '==' }
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @devicefarmer/stf@3.6.1 prepublish: `bower install && not-in-install && gulp build || in-install`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @devicefarmer/stf@3.6.1 prepublish script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/stf/.npm/_logs/2022-03-03T22_01_21_932Z-debug.log
The command '/bin/sh -c export DEBIAN_FRONTEND=noninteractive &&     echo '--- Updating repositories' &&     apt-get update &&     echo '--- Building node' &&     apt-get -y install wget python build-essential cmake &&     cd /tmp &&     wget --progress=dot:mega       https://nodejs.org/dist/v8.9.3/node-v8.9.3-linux-armv7l.tar.xz &&     tar -xJf node-v*.tar.xz --strip-components 1 -C /usr/local &&     rm node-v*.tar.xz &&     useradd --system       --create-home       --shell /usr/sbin/nologin       stf &&     su stf -s /bin/bash -c '/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js install' &&     apt-get -y install --no-install-recommends libzmq3-dev libprotobuf-dev git graphicsmagick yasm &&     echo '--- Building app' &&     mkdir -p /app &&     chown -R stf:stf /tmp/build &&     set -x &&     cd /tmp/build &&     export PATH=$PWD/node_modules/.bin:$PATH &&     sed -i'' -e '/phantomjs/d' package.json &&     npm config set unsafe-perm true &&     echo 'npm cache clean --force' | su stf -s /bin/bash &&     echo 'npm install --no-optional --legacy-peer-deps --loglevel http' | su stf -s /bin/bash &&     echo '--- Assembling app' &&     echo 'npm pack' | su stf -s /bin/bash &&     tar xzf devicefarmer-stf-*.tgz --strip-components 1 -C /app &&     echo '/tmp/build/node_modules/.bin/bower cache clean' | su stf -s /bin/bash &&     echo 'npm prune --production' | su stf -s /bin/bash &&     mv node_modules /app &&     chown -R root:root /app &&     echo '--- Cleaning up' &&     echo 'npm cache clean --force' | su stf -s /bin/bash &&     rm -rf ~/.node-gyp &&     apt-get -y purge wget python build-essential &&     apt-get -y clean &&     apt-get -y autoremove &&     rm -rf /var/cache/apt/* /var/lib/apt/lists/* &&     cd /app &&     rm -rf /tmp/*' returned a non-zero code: 1

Am I the only one?

Thanks.

denis99999 commented 2 years ago

@gounthar ,

I recently built a 3.6.1 without issue on a Raspberry Pi4, using the Dockerfile Dockerfile-debian-armhf, is this your case? Regards.

gounthar commented 2 years ago

Thank you @denis99999 . My Raspberry Pi 3B+ are running Armbian, so 64 bits. I'm building on an Orange Pi Zero, so a true 32 bits arm machine. Did you build from master, or from the 3.6.1 source file release?

When I switch to the 3.6.1 release source files, I get another error:

Processing triggers for libc-bin (2.24-11+deb9u4) ...
--- Building app
+ cd /tmp/build
+ export PATH=/tmp/build/node_modules/.bin:/app/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ sed -i -e /phantomjs/d package.json
+ npm config set unsafe-perm true
+ npm install -g npm
/usr/local/bin/npm -> /usr/local/lib/node_modules/npm/bin/npm-cli.js
/usr/local/bin/npx -> /usr/local/lib/node_modules/npm/bin/npx-cli.js
+ npm@8.5.3
added 149 packages, removed 418 packages and updated 58 packages in 85.128s
+ echo npm cache clean --force
+ su stf -s /bin/bash
npm does not support Node.js v9.9.0
You should probably upgrade to a newer version of node as we
can't make any promises that npm will work with this version.
You can find the latest version at https://nodejs.org/
/usr/local/lib/node_modules/npm/lib/npm.js:32
  #unloaded = false
  ^

SyntaxError: Invalid or unexpected token
    at new Script (vm.js:51:7)
    at createScript (vm.js:136:10)
    at Object.runInThisContext (vm.js:197:10)
    at Module._compile (module.js:613:28)
    at Object.Module._extensions..js (module.js:660:10)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:501:12)
    at Function.Module._load (module.js:493:3)
    at Module.require (module.js:593:17)
    at require (internal/module.js:11:18)
The command '/bin/sh -c export DEBIAN_FRONTEND=noninteractive &&     echo '--- Updating repositories' &&     apt-get update &&     echo '--- Building node' &&     apt-get -y install wget python build-essential cmake &&     cd /tmp &&     wget --progress=dot:mega       https://nodejs.org/dist/v9.9.0/node-v9.9.0-linux-armv7l.tar.xz &&     tar -xJf node-v*.tar.xz --strip-components 1 -C /usr/local &&     rm node-v*.tar.xz &&     useradd --system       --create-home       --shell /usr/sbin/nologin       stf &&     su stf -s /bin/bash -c '/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js install' &&     apt-get -y install --no-install-recommends libzmq3-dev libprotobuf-dev git graphicsmagick yasm &&     echo '--- Building app' &&     mkdir -p /app &&     chown -R stf:stf /tmp/build &&     set -x &&     cd /tmp/build &&     export PATH=$PWD/node_modules/.bin:$PATH &&     sed -i'' -e '/phantomjs/d' package.json &&     npm config set unsafe-perm true &&     npm install -g npm &&     echo 'npm cache clean --force' | su stf -s /bin/bash &&     echo 'npm install --no-optional --loglevel http' | su stf -s /bin/bash &&     echo '--- Assembling app' &&     echo 'npm pack' | su stf -s /bin/bash &&     tar xzf devicefarmer-stf-*.tgz --strip-components 1 -C /app &&     echo '/tmp/build/node_modules/.bin/bower cache clean' | su stf -s /bin/bash &&     echo 'npm prune --production' | su stf -s /bin/bash &&     mv node_modules /app &&     chown -R root:root /app &&     echo '--- Cleaning up' &&     echo 'npm cache clean --force' | su stf -s /bin/bash &&     rm -rf ~/.node-gyp &&     apt-get -y purge wget python build-essential &&     apt-get -y clean &&     apt-get -y autoremove &&     rm -rf /var/cache/apt/* /var/lib/apt/lists/* &&     cd /app &&     rm -rf /tmp/*' returned a non-zero code: 1
denis99999 commented 2 years ago

@gounthar ,

I just built from master and it works well, I recommend using a Raspberry Pi4 for the build because it's much more powerful, it only takes a few minutes to build the docker image, on my side I use a Raspbian lite OS on my Pi whether for construction or production and everything is very stable, but it's true slightly more expensive than a Pi Zero, even if this cost is ridiculous compared to the purchase of mobiles that you have to connect to it, with the advantage that the Pi4 has 32 USB channels against only 8 for a Pi3.

Regards.

gounthar commented 2 years ago

@denis99999 what is the command you're using? Any parameter? What are your kernel and Docker versions?

I'm using Docker version 20.10.5+dfsg1, build 55c4c88 on a Debian 11 bullseye running 5.10.60 armv7l kernel.

denis99999 commented 2 years ago

@gounthar ,

Docker version: 20.10.12 build e91ed57 Kernel: 5.10.63-v7l+ under RaspiOS Busterupdated Command line for STF: docker build -f Dockerfile-debian-armhf -t stf-armv7l:latest .

chiquilindebachinar commented 1 year ago

Hi @denis99999 I'm running DeviceFarmer from the main branch on a few servers, but I'm having issues building it on Raspberry PI 4. Have you tried this lately or are you aware of any issues? Below is the issue I'm experiencing. I appriciete you can point me in any direction.

Model : Raspberry Pi 4 Model B Rev 1.1 Hardware : BCM2835

@raspberrypi:~/stf $ uname -a Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

@raspberrypi:~/stf $ cat cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)"

@raspberrypi:~/stf $ docker -v Docker version 24.0.6, build ed223bc

raspberrypi:~/stf $ sudo docker build -f Dockerfile-debian-armhf -t stf-armv7l:latest .

[+] Building 567.4s (8/8) FINISHED docker:default => [internal] load .dockerignore 0.1s => => transferring context: 182B 0.0s => [internal] load build definition from Dockerfile-debian-armhf 0.1s => => transferring dockerfile: 2.42kB 0.0s => [internal] load metadata for docker.io/arm32v7/debian:buster-slim 1.5s => [internal] load build context 1.1s => => transferring context: 11.14MB 0.9s => [1/4] FROM docker.io/arm32v7/debian:buster-slim@sha256:6a596f170ded1466 17.7sb => => resolve docker.io/arm32v7/debian:buster-slim@sha256:6a596f170ded1466e 0.0sb => => sha256:6a596f170ded1466e2a2b6cbd3630605a36fc9d6a0ac042c1a 529B / 529B 0.0sB => => sha256:d8a196cd840523d75c771d5b697ae4ae72bb560ba0244a 1.48kB / 1.48kB 0.0s8 => => sha256:7e54f06c680feaf82a31a34e7453d765f332f671cfe 22.80MB / 22.80MB 13.6s8 => => extracting sha256:7e54f06c680feaf82a31a34e7453d765f332f671cfecacb622c 3.6s1 => [2/4] WORKDIR /app 3.0s => [3/4] COPY . /tmp/build/ 1.3s => ERROR [4/4] RUN export DEBIAN_FRONTEND=noninteractive && echo '--- 543.3ss ...

.... 61 | >>> rm -rf /tmp/* 62 |

ERROR: failed to solve: process "/bin/sh -c export DEBIAN_FRONTEND=noninteractive & & echo '--- Updating repositories' && apt-get update && echo '--- Build ing node' && apt-get -y install wget python3 build-essential cmake && cd /t mp && wget --progress=dot:mega https://nodejs.org/dist/v17.9.0/node-v17.9 .0-linux-armv7l.tar.xz && tar -xJf node-v.tar.xz --strip-components 1 -C /usr/ local && rm node-v.tar.xz && useradd --system --create-home -- shell /usr/sbin/nologin stf && su stf -s /bin/bash -c '/usr/local/lib/nod e_modules/npm/node_modules/node-gyp/bin/node-gyp.js install' && apt-get -y inst all --no-install-recommends libzmq3-dev libprotobuf-dev git graphicsmagick yasm && echo '--- Building app' && mkdir -p /app && chown -R stf:stf /tmp/build && set -x && cd /tmp/build && export PATH=$PWD/node_modules/.bin:$PATH && sed -i'' -e '/phantomjs/d' package.json && echo 'npm install --python=\ "/usr/bin/python3\" --no-optional --loglevel http' | su stf -s /bin/bash && ech o '--- Assembling app' && echo 'npm pack' | su stf -s /bin/bash && tar xzf devicefarmer-stf-.tgz --strip-components 1 -C /app && echo '/tmp/build/node_mo dules/.bin/bower cache clean' | su stf -s /bin/bash && echo 'npm prune --produc tion' | su stf -s /bin/bash && mv node_modules /app && chown -R root:root / app && echo '--- Cleaning up' && echo 'npm cache clean --force' | su stf -s /bin/bash && rm -rf ~/.node-gyp && apt-get -y purge wget python3 build-ess ential && apt-get -y clean && apt-get -y autoremove && rm -rf /var/cach e/apt/ /var/lib/apt/lists/ && cd /app && rm -rf /tmp/" did not complete successfully: exit code: 1

denis99999 commented 1 year ago

@chiquilindebachinar , I didn't try with a RaspiOS bullseye, but using a Buster on Raspberry Pi4 it works well, let me some time to investigate about that!

chiquilindebachinar commented 1 year ago

Thanks @denis99999 . I was able to build the STF image with Buster, but now Im getting other errors with ADB and Rethinkdb images. I will spend some time analyzing the issues and will share here the outcome.

denis99999 commented 1 year ago

@chiquilindebachinar , I successed to build a docker image of STF using raspberry PI OS Lite (64 bits) based on Debian Bullseye, so the same of yours! Are you behind a proxy ?

chiquilindebachinar commented 1 year ago

Hi @denis99999 I tried using Pi OS Lite (64 bits) based on Debian Bullseye, and now I'm able to build STF. I'm not behind a proxy, so maybe it was a timeout or something related to the other Pi OS. Now, I have an issue with the ADB container which keeps restarting. Have you encountered this issue? It seems there is a compatibility issue with the Raspberry Pi's architecture.

First, I built the STF image, and then I updated the docker-compose to use it.

@raspberrypi:~/stf $docker build -t stf-arm:latest -f Dockerfile-stf-debian9-armhf .

raspberrypi:~/stf $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 92eaae1aac71 stf-armv7l:latest "stf local --adb-hos…" 5 minutes ago Up 5 minutes 0.0.0.0:7100->7100/tcp, 0.0.0.0:7110->7110/tcp, 0.0.0.0:7400-7500->7400-7500/tcp stf 66287380fc1e devicefarmer/adb:latest "/sbin/tini -- adb -…" 47 minutes ago Restarting (1) 23 seconds ago adb fea1b3ee8c5f rethinkdb:2.4.2 "rethinkdb --bind al…" 2 hours ago Up 31 minutes 8080/tcp, 28015/tcp, 29015/tcp

@raspberrypi:~/stf $ docker logs adb standard_init_linux.go:219: exec user process caused: exec format error standard_init_linux.go:219: exec user process caused: exec format error standard_init_linux.go:219: exec user process caused: exec format error standard_init_linux.go:219: exec user process caused: exec format error

@raspberrypi:~/stf $ cat docker-compose.yaml version: "3"

services: rethinkdb: container_name: rethinkdb image: rethinkdb:2.4.2 restart: unless-stopped volumes:

volumes: rethinkdb-data: {}

@raspberrypi:~ $ uname -a Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux

@raspberrypi:~ $ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

chiquilindebachinar commented 1 year ago

Maybe not relevant, but I tried using gounthar/docker-adb:docker-adb-aarch64 image, and it's working okay. @raspberrypi:~ $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES df4e5fa1c93d gounthar/docker-adb:docker-adb-aarch64 "/sbin/tini -- adb -…" 9 minutes ago Up 3 minutes 5037/tcp adb 92eaae1aac71 stf-armv7l:latest "stf local --adb-hos…" 53 minutes ago Up 3 minutes 0.0.0.0:7100->7100/tcp, 0.0.0.0:7110->7110/tcp, 0.0.0.0:7400-7500->7400-7500/tcp stf fea1b3ee8c5f rethinkdb:2.4.2 "rethinkdb --bind al…" 3 hours ago Up 3 minutes 8080/tcp, 28015/tcp, 29015/tcp rethinkdb

denis99999 commented 1 year ago

@chiquilindebachinar , what rethinkdb image do you use ? Or how did you build it (Dockerfile) ? I mean your rethinkdb database is running on Raspberry Pi4 64 bits also ?

im-0 commented 1 year ago

At least partially fixed by https://github.com/DeviceFarmer/stf/pull/712.

@chiquilindebachinar, @denis99999, just to share my experience with similar issues:

I am using stf on a Raspberry Pi 4 with Fedora. In my case, it was much easier to install adb on the host system from a standard Fedora repository.

RethinkDB is trickier, but I was able to package it for Fedora: https://github.com/im-0/fedora-rpm.rethinkdb. Though I do not recommend this if you do not have any experience in creating new packages for your preferred Linux distro, just modifying someone else's Dockerfile might be easier =).

denis99999 commented 1 year ago

@im-0 , there is no problem for building an ARM 32/64 bit binaries of rethinkDB as docker images, but I meet an issue on running such images on the latest Raspberry PI OS (Bullseye Kernel version 6.1, 32 or 64 bit, Aarch64 architecture), see #7126 for details

im-0 commented 1 year ago

@denis99999 I see armv7l in your log. It is a 32-bit ARM ISA.

Have you verified that your rethinkdb binary is actually 64-bit?

Also, try to build RethinkDB natively, without docker. It should be relatively easy as it does not have that many dependencies. As I said, I use natively built aarch64 RethinkDB running on aarch64 Linux and everything works just fine.

denis99999 commented 1 year ago

@im-0 , I closed the rethinkdb ticket, it works well now with 64-bit rethinkdb binaries (my docker installation on Raspberry Pi OS was not adequate)