search

AVENTER-UG / docker-matrix

docker image for matrix.org
https://riot.im/app/#/room/#dockermatrix:matrix.aventer.biz
GNU General Public License v2.0
93 stars 19 forks source link

Support for power architecture #44

Closed junawaneshivani closed 3 years ago

junawaneshivani commented 4 years ago

Hi,

I was able to build and run the docker-matrix container on power successfully with some minor changes to the Dockerfile. Will you be willing to support power architecture ?

andreaspeters commented 4 years ago

Hi @junawaneshivani . We are ever willing to support new architectures. :-) But it depends how easy it is for us to maintain it. Can u give as a link to the changes?

seth-priya commented 4 years ago

@andreaspeters (cc @junawaneshivani ) , thanks for your interest and sorry this took so long - we were awaiting clearance on our side for code contribution.

So here are the diffs, nothing specific to Power per se, just adding multiarch support to the dockerfile makes it compile and clear the basic validation 

--- a/Dockerfile
+++ b/Dockerfile
@@ -17,6 +17,9 @@ ARG BV_SYN=master
 ARG BV_TUR=master
 ARG TAG_SYN=v1.3.1

+# target architecture
+ARG ARCH=amd64
+
 # user configuration
 ENV MATRIX_UID=991 MATRIX_GID=991

@@ -31,7 +34,7 @@ RUN set -ex \
     && apt-get clean \
     && apt-get update -y -q --fix-missing\
     && apt-get upgrade -y \
-    && buildDeps=' \
+    && buildDeps=" \
         file \
         gcc \
         git \
@@ -46,13 +49,13 @@ RUN set -ex \
         libtool \
         libxml2-dev \
         libxslt1-dev \
-        linux-headers-amd64 \
+        linux-headers-${ARCH} \
         make \
         zlib1g-dev \
         python3-dev \
         python3-setuptools \
         libpq-dev \
-    ' \
+    " \
     && apt-get install -y --no-install-recommends \
         $buildDeps \
         bash \
@@ -91,4 +94,4 @@ RUN set -ex \
     && rm -rf /synapse

 USER matrix
-ENV LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so.2"
+ENV LD_PRELOAD="/usr/lib/${ARCH}-linux-gnu/libjemalloc.so.2"

I'll raise a PR shortly so that easier for you to review, thank you!

andreaspeters commented 4 years ago

The docker image "debian:buster" does not have the linux header files for powerpc:

seth-priya commented 4 years ago

so, these are not sufficient?

root@6405bb9d6c59:/# apt-get install linux-headers-powerpc64le
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-common binutils-powerpc64le-linux-gnu cpp-8 gcc-8 libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libgcc-8-dev libgomp1 libisl19
  libitm1 liblsan0 libmpc3 libmpfr6 libquadmath0 libssl1.1 libtsan0 libubsan1 linux-headers-4.19.0-10-common linux-headers-4.19.0-10-powerpc64le linux-kbuild-4.19
  linux-libc-dev manpages manpages-dev
Suggested packages:
  binutils-doc gcc-8-locales gcc-8-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libmpx2-dbg
  libquadmath0-dbg glibc-doc man-browser
The following NEW packages will be installed:
  binutils binutils-common binutils-powerpc64le-linux-gnu cpp-8 gcc-8 libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libgcc-8-dev libgomp1 libisl19
  libitm1 liblsan0 libmpc3 libmpfr6 libquadmath0 libssl1.1 libtsan0 libubsan1 linux-headers-4.19.0-10-common linux-headers-4.19.0-10-powerpc64le
  linux-headers-powerpc64le linux-kbuild-4.19 linux-libc-dev manpages manpages-dev
0 upgraded, 29 newly installed, 0 to remove and 0 not upgraded.
Need to get 47.2 MB of archives.
After this operation, 204 MB of additional disk space will be used.
andreaspeters commented 4 years ago

Strange... 

apt-get install linux-headers-powerpc64le
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package linux-headers-powerpc64le is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'linux-headers-powerpc64le' has no installation candidate
cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Which docker image do u use?

seth-priya commented 4 years ago

yeah

root@4a9ae944d2e9:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

and I have this image debian buster db6f732df0b5 12 days ago 153MB

just pulled it again to be sure that I have the latest docker pull debian:buster

and I have the same on buster-slim as well

$ docker pull debian:buster-slim
buster-slim: Pulling from library/debian
f8aef3d2247e: Pull complete
Digest: sha256:b527bec3708ee1fe2ffb4625d742a60bb45b06064e7c64a81b16550fe42710f5
Status: Downloaded newer image for debian:buster-slim
docker.io/library/debian:buster-slim
[user3@p006vm77 docker-matrix]$ docker run -it debian:buster-slim bash
root@3fe3d57251f2:/# apt-get update
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:2 http://deb.debian.org/debian buster InRelease [122 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://security.debian.org/debian-security buster/updates/main ppc64el Packages [222 kB]
Get:5 http://deb.debian.org/debian buster/main ppc64el Packages [7692 kB]
Get:6 http://deb.debian.org/debian buster-updates/main ppc64el Packages [7868 B]
Fetched 8162 kB in 4s (1847 kB/s)
Reading package lists... Done
root@3fe3d57251f2:/# apt-get install linux-headers-powerpc64le
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-common binutils-powerpc64le-linux-gnu cpp-8 gcc-8 libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libelf1 libgcc-8-dev libgomp1
  libisl19 libitm1 liblsan0 libmpc3 libmpfr6 libquadmath0 libssl1.1 libtsan0 libubsan1 linux-headers-4.19.0-10-common linux-headers-4.19.0-10-powerpc64le
  linux-kbuild-4.19 linux-libc-dev manpages manpages-dev
Suggested packages:
  binutils-doc gcc-8-locales gcc-8-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg libmpx2-dbg
  libquadmath0-dbg glibc-doc man-browser
The following NEW packages will be installed:
  binutils binutils-common binutils-powerpc64le-linux-gnu cpp-8 gcc-8 libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0 libelf1 libgcc-8-dev libgomp1
  libisl19 libitm1 liblsan0 libmpc3 libmpfr6 libquadmath0 libssl1.1 libtsan0 libubsan1 linux-headers-4.19.0-10-common linux-headers-4.19.0-10-powerpc64le
  linux-headers-powerpc64le linux-kbuild-4.19 linux-libc-dev manpages manpages-dev
0 upgraded, 30 newly installed, 0 to remove and 0 not upgraded.
Need to get 47.4 MB of archives.
After this operation, 206 MB of additional disk space will be used.
Do you want to continue? [Y/n]

debian buster-slim 46f79af554ef 12 days ago 106MB

andreaspeters commented 4 years ago

Ah ok, so u use the "http://deb.debian.org/debian buster/main ppc64el". Now I understand! I have to change the "FROM" line to FROM ${ARCH}/debian:buster-slim.

andreaspeters commented 4 years ago

But its still not working:

standard_init_linux.go:219: exec user process caused: exec format error

Here is the whole build, maybe u can compare it with your one? 

docker build -t avhost/docker-matrix:latest --build-arg ARCH=ppc64le . 
Sending build context to Docker daemon  49.66kB
Step 1/16 : ARG ARCH=amd64
Step 2/16 : FROM ${ARCH}/debian:buster-slim
buster-slim: Pulling from ppc64le/debian
f8aef3d2247e: Pull complete 
Digest: sha256:75764b9bb342b496015066f99097a4cadaa0125cb3d175f1969e0f597c83913c
Status: Downloaded newer image for ppc64le/debian:buster-slim
 ---> 46f79af554ef
Step 3/16 : MAINTAINER Andreas Peters <support@aventer.biz>
 ---> Running in 2abf0e482879
Removing intermediate container 2abf0e482879
 ---> 3b0223c63000
Step 4/16 : COPY adds/start.sh /start.sh
 ---> 687fb27826b4
Step 5/16 : ENTRYPOINT ["/start.sh"]
 ---> Running in 05602c6dd252
Removing intermediate container 05602c6dd252
 ---> 3b56992df7d1
Step 6/16 : CMD ["autostart"]
 ---> Running in baa7fa81e4e4
Removing intermediate container baa7fa81e4e4
 ---> bf3cad78a4e6
Step 7/16 : EXPOSE 8448
 ---> Running in 94f6c6a05fa2
Removing intermediate container 94f6c6a05fa2
 ---> c734e892a458
Step 8/16 : VOLUME ["/data"]
 ---> Running in e9481c1101bb
Removing intermediate container e9481c1101bb
 ---> cc50fd2e4c3f
Step 9/16 : ARG BV_SYN=release-v1.19.3
 ---> Running in 982410fcc123
Removing intermediate container 982410fcc123
 ---> 9249b34404f4
Step 10/16 : ARG BV_TUR=master
 ---> Running in 5d98a9b00299
Removing intermediate container 5d98a9b00299
 ---> 7593872cced1
Step 11/16 : ARG TAG_SYN=v1.19.3
 ---> Running in d516685f2401
Removing intermediate container d516685f2401
 ---> e59dcc4a009b
Step 12/16 : ENV MATRIX_UID=991 MATRIX_GID=991
 ---> Running in 4e9e8486d6bb
Removing intermediate container 4e9e8486d6bb
 ---> 301496ed567d
Step 13/16 : ARG REBUILD=1
 ---> Running in c74cae396100
Removing intermediate container c74cae396100
 ---> 914f54281886
Step 14/16 : RUN set -ex     && mkdir /uploads     && export DEBIAN_FRONTEND=noninteractive     && mkdir -p /var/cache/apt/archives     && touch /var/cache/apt/archives/lock     && apt-get clean     && apt-get update -y -q --fix-missing    && apt-get upgrade -y     && buildDeps="         file         gcc         git         libevent-dev         libffi-dev         libgnutls28-dev         libjpeg62-turbo-dev         libldap2-dev         libsasl2-dev         libsqlite3-dev         libssl-dev         libtool         libxml2-dev         libxslt1-dev         linux-headers-${ARCH}         make         zlib1g-dev         python3-dev         python3-setuptools         libpq-dev     "     && apt-get install -y --no-install-recommends         $buildDeps         bash         coreutils         coturn         libffi6         libjpeg62-turbo         libssl1.1         libtool         libxml2         libxslt1.1         pwgen         python3         python3-pip         python3-jinja2         sqlite         libjemalloc2         zlib1g     ;     pip3 install --upgrade wheel ;    pip3 install --upgrade psycopg2;    pip3 install --upgrade python-ldap ;    pip3 install --upgrade twisted==20.3.0 ;    pip3 install --upgrade redis ;    pip3 install -e "git+https://github.com/t2bot/synapse-simple-antispam#egg=synapse-simple-antispam" ;    pip3 install -e "git+https://github.com/matrix-org/mjolnir.git#egg=mjolnir&subdirectory=synapse_antispam" ;    pip3 install --upgrade lxml     ;     groupadd -r -g $MATRIX_GID matrix     && useradd -r -d /data -M -u $MATRIX_UID -g matrix matrix     && chown -R $MATRIX_UID:$MATRIX_GID /data     && chown -R $MATRIX_UID:$MATRIX_GID /uploads     && git clone --branch $BV_SYN --depth 1 https://github.com/matrix-org/synapse.git     && cd /synapse     && git checkout -b tags/$TAG_SYN     && pip3 install --upgrade .[all]     && GIT_SYN=$(git ls-remote https://github.com/matrix-org/synapse $BV_SYN | cut -f 1)     && echo "synapse: $BV_SYN ($GIT_SYN)" >> /synapse.version     && cd /     && rm -rf /synapse     && mkdir -p /usr/lib/${ARCH}-linux-gnu     && ln -s /usr/lib/x86_64-linux-gnu/libjemalloc.so.2 /usr/lib/${ARCH}-linux-gnu/libjemalloc.so.2     ;     apt-get autoremove -y $buildDeps ;     apt-get autoremove -y ;    rm -rf /var/lib/apt/* /var/cache/apt/*
 ---> Running in 70c80b2b644a
standard_init_linux.go:219: exec user process caused: exec format error
The command '/bin/sh -c set -ex     && mkdir /uploads     && export DEBIAN_FRONTEND=noninteractive     && mkdir -p /var/cache/apt/archives     && touch /var/cache/apt/archives/lock     && apt-get clean     && apt-get update -y -q --fix-missing    && apt-get upgrade -y     && buildDeps="         file         gcc         git         libevent-dev         libffi-dev         libgnutls28-dev         libjpeg62-turbo-dev         libldap2-dev         libsasl2-dev         libsqlite3-dev         libssl-dev         libtool         libxml2-dev         libxslt1-dev         linux-headers-${ARCH}         make         zlib1g-dev         python3-dev         python3-setuptools         libpq-dev     "     && apt-get install -y --no-install-recommends         $buildDeps         bash         coreutils         coturn         libffi6         libjpeg62-turbo         libssl1.1         libtool         libxml2         libxslt1.1         pwgen         python3         python3-pip         python3-jinja2         sqlite         libjemalloc2         zlib1g     ;     pip3 install --upgrade wheel ;    pip3 install --upgrade psycopg2;    pip3 install --upgrade python-ldap ;    pip3 install --upgrade twisted==20.3.0 ;    pip3 install --upgrade redis ;    pip3 install -e "git+https://github.com/t2bot/synapse-simple-antispam#egg=synapse-simple-antispam" ;    pip3 install -e "git+https://github.com/matrix-org/mjolnir.git#egg=mjolnir&subdirectory=synapse_antispam" ;    pip3 install --upgrade lxml     ;     groupadd -r -g $MATRIX_GID matrix     && useradd -r -d /data -M -u $MATRIX_UID -g matrix matrix     && chown -R $MATRIX_UID:$MATRIX_GID /data     && chown -R $MATRIX_UID:$MATRIX_GID /uploads     && git clone --branch $BV_SYN --depth 1 https://github.com/matrix-org/synapse.git     && cd /synapse     && git checkout -b tags/$TAG_SYN     && pip3 install --upgrade .[all]     && GIT_SYN=$(git ls-remote https://github.com/matrix-org/synapse $BV_SYN | cut -f 1)     && echo "synapse: $BV_SYN ($GIT_SYN)" >> /synapse.version     && cd /     && rm -rf /synapse     && mkdir -p /usr/lib/${ARCH}-linux-gnu     && ln -s /usr/lib/x86_64-linux-gnu/libjemalloc.so.2 /usr/lib/${ARCH}-linux-gnu/libjemalloc.so.2     ;     apt-get autoremove -y $buildDeps ;     apt-get autoremove -y ;    rm -rf /var/lib/apt/* /var/cache/apt/*' returned a non-zero code: 1
seth-priya commented 4 years ago

so couple of things,

1- I don't need to change the FROM line because the image uses a multiarch manifest, which essentially means that it will pull the correct image based on the architecture on which or from which you issuing your "docker pull" or "docker run" command. Therefore "FROM Debian:buster-slim" works just fine as in my example above on ppc64le. 2- The build-arg "ARCH" should be set to "powerpc64le" and not "ppc64le" as mentioned in the PR because that's the naming convention followed for the linux-headers package on Power, that is, it is linux-headers-powerpc64le 3- The exec format error typically comes when there is an architecture mismatch between the host and container or host and binary. So for example, the powerpc64le/ppc64le image can be built and run only on a Power PC host/VM/machine.

this is from my host VM

$ uname -m
ppc64le
$ uname -a
Linux p006vm77 3.10.0-1127.10.1.el7.ppc64le #1 SMP Tue May 26 15:11:07 EDT 2020 ppc64le ppc64le ppc64le GNU/Linux

and this is my build command docker build -t avhost/docker-matrix:latest --build-arg ARCH=powerpc64le .

the dockerfile is the same as was in my PR commit

Hope that helps a bit?

seth-priya commented 4 years ago

Please do let me know if I can help with any additional testing or anything else, thanks!

andreaspeters commented 4 years ago

Yeah I belief it's working for you. But not for me, like I show you. :-) Well, I think the building problems is only because I do not have a powerpc client to test it. I can keep the Dockerfile like it is if its enough for you. But then u have to build your own docker image. Actually, my claim was to give u also a docker image with every synapse update. :-) I will see if I can build a powerpc one via travis-ci.

seth-priya commented 4 years ago

thanks @andreaspeters - that would be most awesome and yes, travis-ci would definitely be a good way forward on this and in case you need a PowerPC VM for testing on your side, you should be able to request one from here https://osuosl.org/services/powerdev/request_hosting/ If you do, please use "Gerrit Huizenga" as the name of the "IBM Advocate" , @gerrith3, FYI

andreaspeters commented 4 years ago

your welcome @seth-priya :-) and thanks for the link.

I could build a ppc64le version via travis-ci. if u have time, can you have a look if the image is working (means, if synapse is starting and usable)? you will find it with the tag "ppc". https://hub.docker.com/r/avhost/docker-matrix/tags?page=1

junawaneshivani commented 4 years ago

@andreaspeters I was trying to validate avhost/docker-matrix:ppc image.. However I see below errors.. Seems like the LD_PRELOAD path is not getting set properly...

#  docker run -v /opt/synapse:/data  --rm --user 991:991 -e SERVER_NAME=localhost -e REPORT_STATS=no avhost/docker-matrix:ppc generate
-=> generate turn config
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
-=> generate synapse config
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
/usr/bin/python3: Error while finding module specification for 'synapse.app.homeserver' (ModuleNotFoundError: No module named 'synapse')
-=> configure some settings in homeserver.yaml
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
awk: cannot open /data/homeserver.tmp (No such file or directory)
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib/libjemalloc.so.2' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
sed: can't read /data/localhost.log.config: No such file or directory
-=> configure some settings in localhost.log.config

-=> you have to review the generated configuration file homeserver.yaml
seth-priya commented 4 years ago

I think these lines were added yesterday, could it be related? https://github.com/AVENTER-UG/docker-matrix/blob/f23a915c30613ec873fac5c7891707d2da9d7722/Dockerfile#L99

we are hard coding the arch on LHS ...

andreaspeters commented 4 years ago

I was using "docker buildx", but thats not really working under travis-ci. The docker images was always not working. U already could see it at the image size. Much smaller then it should be. If I use it at my local client, the images was fine with "docker buildx". Whatever... :-) I changed back to the normal "docker build" together with the build-arg. Please have a look if these image works for you. The tag is "ppc64le-ppc" (https://hub.docker.com/repository/docker/avhost/docker-matrix/tags?page=1&name=ppc).

junawaneshivani commented 4 years ago

I was able to run the new image with tag ppc64le-ppc. Pasting validation logs below...

# docker run -v /opt/synapse:/data  --rm --user 991:991 -e SERVER_NAME=localhost -e REPORT_STATS=no avhost/docker-matrix:ppc64le-ppc generate
-=> generate turn config
-=> generate synapse config
Generating config file /data/homeserver.tmp
Generating log config file /data/localhost.log.config which will log to /homeserver.log
Generating signing key file /data/localhost.signing.key
A config file has been generated in '/data/homeserver.tmp' for server name 'localhost'. Please review this file and customise it to your needs.
-=> configure some settings in homeserver.yaml
-=> configure some settings in localhost.log.config

-=> you have to review the generated configuration file homeserver.yaml
# ls /opt/synapse/
homeserver.yaml  localhost.log.config  localhost.signing.key  turnserver.conf

# echo "userdb=/data/turndb" >> /opt/synapse/turnserver.conf
# docker run --name=matrix -it --rm -p 8448:8448 -p 8008:8008 -p 3478:3478 -p 3478:3478/udp -p 5349:5349/udp -p 5349:5349 -p 49152-49300:49152-49300/udp -p 49152-49300:49152-49300 -v /opt/synapse:/data avhost/docker-matrix:ppc64le-ppc start

-=> start turn
-=> start matrix
0: log file opened: /var/tmp/turn_8_2020-09-23.log
0:
RFC 3489/5389/5766/5780/6062/6156 STUN/TURN Server
Version Coturn-4.5.1.1 'dan Eider'
0:
Max number of open files/sockets allowed for this process: 1048576
0:
Due to the open files/sockets limitation,
max supported number of TURN Sessions possible is: 524000 (approximately)
0:

==== Show him the instruments, Practical Frost: ====

0: TLS supported
0: DTLS supported
0: DTLS 1.2 supported
0: TURN/STUN ALPN supported
0: Third-party authorization (oAuth) supported
0: GCM (AEAD) supported
0: OpenSSL compile-time version: OpenSSL 1.1.1d  10 Sep 2019 (0x1010104f)
0:
0: SQLite supported, default database location is /var/lib/turn/turndb
0: Redis supported
0: PostgreSQL supported
0: MySQL supported
0: MongoDB is not supported
0:
0: Default Net Engine version: 3 (UDP thread per CPU core)

=====================================================

0: Domain name:
0: Default realm: turn.localhost
0:
CONFIGURATION ALERT: You specified --lt-cred-mech and --use-auth-secret in the same time.
Be aware that you could not mix the username/password and the shared secret based auth methohds.
Shared secret overrides username/password based auth method. Check your configuration!
0: ERROR:
CONFIG ERROR: Empty cli-password, and so telnet cli interface is disabled! Please set a non empty cli-password!
0: WARNING: cannot find certificate file: /data/localhost.tls.crt (1)
0: WARNING: cannot start TLS and DTLS listeners because certificate file is not set properly
0: WARNING: cannot find private key file: /data/localhost.tls.key (1)
0: WARNING: cannot start TLS and DTLS listeners because private key file is not set properly
0: WARNING: cannot find DH key file: /data/localhost.tls.dh (1)
0: NO EXPLICIT LISTENER ADDRESS(ES) ARE CONFIGURED
0: ===========Discovering listener addresses: =========
0: Listener address to use: 127.0.0.1
0: Listener address to use: 172.17.0.5
0: =====================================================
0: Total: 1 'real' addresses discovered
0: =====================================================
0: NO EXPLICIT RELAY ADDRESS(ES) ARE CONFIGURED
0: ===========Discovering relay addresses: =============
0: Relay address to use: 172.17.0.5
0: =====================================================
0: Total: 1 relay addresses discovered
0: =====================================================
Cannot create pid file: /var/run/turnserver.pid: Permission denied
0: Cannot create pid file: /var/run/turnserver.pid
0: pid file created: /var/tmp/turnserver.pid
0: IO method (main listener thread): epoll (with changelist)
0: Wait for relay ports initialization...
0:   relay 172.17.0.5 initialization...
0:   relay 172.17.0.5 initialization done
0: Relay ports initialization done
0: IO method (general relay thread): epoll (with changelist)
0: IO method (general relay thread): epoll (with changelist)
1: turn server id=0 created
1: turn server id=1 created
1: IO method (general relay thread): epoll (with changelist)
1: IO method (general relay thread): epoll (with changelist)
1: turn server id=2 created
1: turn server id=3 created
1: IO method (general relay thread): epoll (with changelist)
1: turn server id=4 created
1: IO method (general relay thread): epoll (with changelist)
1: turn server id=6 created
1: IO method (general relay thread): epoll (with changelist)
1: turn server id=7 created
1: IO method (general relay thread): epoll (with changelist)
1: turn server id=5 created
1: Total General servers: 8
1: IO method (auth thread): epoll (with changelist)
1: IO method (admin thread): epoll (with changelist)
1: IO method (auth thread): epoll (with changelist)
1: IO method (auth thread): epoll (with changelist)
1: IO method (auth thread): epoll (with changelist)
1: SQLite DB connection success: /data/turndb
This server is configured to use 'matrix.org' as its trusted key server via the
'trusted_key_servers' config option. 'matrix.org' is a good choice for a key
server since it is long-lived, stable and trusted. However, some admins may
wish to use another server for this purpose.

To suppress this warning and continue using 'matrix.org', admins should set
'suppress_key_server_warning' to 'true' in homeserver.yaml.
--------------------------------------------------------------------------------
andreaspeters commented 4 years ago

That makes me happy to see. :-) I was also able to create a docker manifest via travis-ci. So, in the future you will get a ppc64le image with every synapse update. :-) Before I will merge it, there is one issue I have to discuss with my partner. I had to remove the "special" malloc lib therefore your build is working.

image

seth-priya commented 4 years ago

thanks @andreaspeters for your support in driving this all the way through 👍

andreaspeters commented 4 years ago

thanks to you @seth-priya and @junawaneshivani. without u both there would not be a powerpc image. :-)

andreaspeters commented 3 years ago

Hi @seth-priya and @junawaneshivani, hope u are doing well specially in these times.

Short question to u both. I create a github workflow to build docker images. Would be nice if you can try out the "docker-matrix:test" image therefore I can see if the ppc building also working. :-)

gerrith3 commented 3 years ago

BTW, @andreaspeters if you need your own Power VM to run/test things on, they are available for open source developers for free from OSU's OSL: https://osuosl.org/services/powerdev/ - you can use me "Gerrit Huizenga" as the IBM sponsor if you choose to do that. I've asked Priya to support the testing as you've requested, though, because it will be good to see if cross builds with qemu are generating valid images. In most cases, they should, and this project seems simple enough in scope that I believe it should work. I do like how you (possibly?) have made github actions work for at least one class of projects (I hope! ;--)).

junawaneshivani commented 3 years ago

@andreaspeters seems like the cross build is not working as expected. I could validate the ppc64le-v1.26.0 tag but test tag (multiarch) is giving exec error even though the architecture is ppc64le.

# docker images
REPOSITORY                                             TAG                                          IMAGE ID                            CREATED             SIZE
avhost/docker-matrix                             ppc64le-v1.26.0                   abe2d480ce3a                  7 hours ago         438MB
avhost/docker-matrix                             test                                           79a5ec33b95a                  11 hours ago        335MB

# docker run -v /opt/synapse:/data --rm -e SERVER_NAME=example.com -e REPORT_STATS=no avhost/docker-matrix:test generate
standard_init_linux.go:211: exec user process caused "exec format error"

# docker image inspect avhost/docker-matrix:test | grep Arch
        "Architecture": "ppc64le",

# docker run -v /opt/synapse:/data --rm -e SERVER_NAME=example.com -e REPORT_STATS=no  avhost/docker-matrix:ppc64le-v1.26.0 generate
-=> generate turn config
-=> generate synapse config
Generating config file /data/homeserver.tmp
Generating log config file /data/example.com.log.config which will log to /homeserver.log
Generating signing key file /data/example.com.signing.key
A config file has been generated in '/data/homeserver.tmp' for server name 'example.com'. Please review this file and customise it to your needs.
-=> configure some settings in homeserver.yaml
-=> configure some settings in example.com.log.config

-=> you have to review the generated configuration file homeserver.yaml
andreaspeters commented 3 years ago

Hi @gerrith3 , thanks for the hin. I applied for the free access. Would be happy to get one.

Hi @junawaneshivani, thanks for testing. :-) The "ppc64le-v1.26.0" is the version build by travis-ci. But since travis-ci do not accept my Synapse repository to migrate to there new platform, I have to move to github actions. Thats the "test" one. So, it looks like, the executable does not have the correct arch. I will have a look.

gerrith3 commented 3 years ago

@andreaspeters why did travis not accept your Synapse repository? Is it open source and are you the maintainer? If so, it should be approved?

andreaspeters commented 3 years ago

@gerrith3 thats a good question. I could migrate every one of my repos only not docker-matrix and docker-matrix-riot. But these also the one who need to be build quite often. :-)

gerrith3 commented 3 years ago

@andreaspeters do you have a support ticket number from travis? I can raise it with them directly and see what's happening.

andreaspeters commented 3 years ago

No, I didn't open a issue but if you have connections to there, then I see a better chance they will do something. :-) So, give me time until Sunday to open a issue.

gerrith3 commented 3 years ago

Sounds good - they may respond to it immediately but if not let me know the issue number and I will follow up on Monday with them.

andreaspeters commented 3 years ago

@gerrith3 @junawaneshivani good news, travis fix sth therefore I can migrate the repository to there .com service. So, the build of ppc64le is working again. :-) And I'm also looking forward to get access to OSL. Maybe there are more projects I can port to ppc64le. :thinking:

gerrith3 commented 3 years ago

@andreaspeters Awesome news! Let us know if you need any help getting on to Power. The OSL request should get processed within a week. I didn't see it come in yet but if you did, we have some folks that review quickly and can provide approvals. If you added my name as your iBM contact they'll ask me direclty and I will approve for hardware at OSL.