Open MTRNord opened 6 months ago
sando38
commented
6 months ago Hello, yes, I planned to build the ARM variant with github runners, but the bug with QEMU still persists. I guess the easiest way is as you said to build yourself. Do you use a MS SQL backend? If not, you may use the Dockerfile I have recently pushed into the repository. In case of questions for build, just let me know +1
MTRNord
commented
6 months ago Hi thanks for the response :) I planned to use postgres as my database as I run already a cluster for it :) I will take a look at that Dockerfile :)
MTRNord
commented
6 months ago Hm I am indeed hitting an issue. On the https://github.com/processone/ejabberd-contrib build step it fails with this error when building the 24 image:
=> [build 2/13] WORKDIR /ejabberd/.ejabberd-modules/sources/ejabberd-contrib 0.0s
=> [build 3/13] RUN git clone https://github.com/processone/ejabberd-contrib --depth 1 . && install mod_s3_upload/src/*.erl /ejabberd/src && install mod_s3_upload/include/*.hrl /ejabberd/include 1.1s
=> [build 4/13] WORKDIR /ejabberd 0.0s
=> ERROR [build 5/13] RUN mv .github/container/ejabberdctl.template . && ./autogen.sh && ./configure --with-rebar=mix --enable-all && make deps && make rel 0.3s
------
> [build 5/13] RUN mv .github/container/ejabberdctl.template . && ./autogen.sh && ./configure --with-rebar=mix --enable-all && make deps && make rel:
0.213 mv: can't rename '.github/container/ejabberdctl.template': No such file or directory
------
Dockerfile:110
--------------------
109 | WORKDIR $BUILD_DIR
110 | >>> RUN mv .github/container/ejabberdctl.template . \
111 | >>> && ./autogen.sh \
112 | >>> && ./configure --with-rebar=mix --enable-all \
113 | >>> && make deps \
114 | >>> && make rel
115 |
--------------------
ERROR: failed to solve: process "/bin/sh -c mv .github/container/ejabberdctl.template . && ./autogen.sh && ./configure --with-rebar=mix --enable-all && make deps && make rel" did not complete successfully: exit code: 1Any ideas on that?
MTRNord
commented
6 months ago I assume this is referring to https://github.com/processone/ejabberd/blob/master/.github/container/ejabberdctl.template ?
MTRNord
commented
6 months ago ah I see what happened. The CI clones this prior to building :)
sando38
commented
6 months ago I have no time right now to double check, but maybe the github action workflow helps you already to show how the image would be built:
https://github.com/sando38/helm-ejabberd/blob/main/.github/workflows/ctr.yaml
MTRNord
commented
6 months ago yep seen that and it worked from what I can tell. Waiting right now for it to deploy itself on the cluster to check if it works too :D Thanks!
MTRNord
commented
6 months ago Hm ok it kinda works apart from elector failing with an syntax error. Should I make a new issue for that?
2024-05-16 17:24:36.343324+00:00 [info] <0.186.0>@ejabberd_config:load/1:89 Configuration loaded successfully
2024-05-16 17:24:36.505360+00:00 [info] <0.440.0>@ejabberd_systemd:init/1:94 Got no NOTIFY_SOCKET, notifications disabled
2024-05-16 17:24:36.603996+00:00 [warning] <0.476.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 100
2024-05-16 17:24:36.604144+00:00 [warning] <0.476.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 5000
2024-05-16 17:24:36.604241+00:00 [warning] <0.476.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 100
2024-05-16 17:24:36.604903+00:00 [warning] <0.476.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 5000
2024-05-16 17:24:36.670394+00:00 [info] <0.439.0>@gen_mod:start_modules/0:141 Loading modules for xmpp.midnightthoughts.space
2024-05-16 17:24:37.277626+00:00 [info] <0.606.0>@mod_stun_disco:parse_listener/1:626 Going to offer STUN service: 100.96.7.239:3478 (udp)
2024-05-16 17:24:37.592657+00:00 [warning] <0.654.0>@ejabberd_captcha:get_prog_name/0:444 The option captcha_cmd is not configured, but some module wants to use the CAPTCHA feature.
2024-05-16 17:24:37.752837+00:00 [info] <0.186.0>@ejabberd_cluster_mnesia:wait_for_sync/1:123 Waiting for Mnesia synchronization to complete
2024-05-16 17:24:37.836868+00:00 [info] <0.186.0>@ejabberd_app:start/2:63 ejabberd 24.2.0 is started in the node 'ejabberd@ejabberd-0.ejabberd-headless.ejabberd.svc.cluster.local' in 2.08s
2024-05-16 17:24:37.838446+00:00 [info] <0.650.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at 100.96.7.239:7777 for mod_proxy65_stream
2024-05-16 17:24:37.838502+00:00 [info] <0.467.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at :5222 for ejabberd_c2s
2024-05-16 17:24:37.838483+00:00 [info] <0.466.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at 127.0.0.1:5281 for ejabberd_http
2024-05-16 17:24:37.838672+00:00 [info] <0.471.0>@ejabberd_listener:init/4:129 Start accepting UDP connections at :3478 for ejabberd_stun
2024-05-16 17:24:37.838617+00:00 [info] <0.469.0>@ejabberd_listener:init/4:163 Start accepting TLS connections at :5443 for ejabberd_http
2024-05-16 17:24:37.838789+00:00 [info] <0.470.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at :5269 for ejabberd_s2s_in
2024-05-16 17:24:37.838811+00:00 [info] <0.468.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at :5280 for ejabberd_http
run.sh: ==> Start elector sidecar service on 127.0.0.1:4040 ...
run.sh: ==> Wait for elector sidecar to be available on 127.0.0.1:4040 ...
nc: missing port number
/usr/local/bin/elector: line 0: syntax error: unexpected word (expecting ")")
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
nc: missing port number
sh: can't kill pid 247: No such processhttps://github.com/MTRNord/cluster/tree/main/apps/base/ejabberd is the config/setup I am running if that helps
sando38
commented
6 months ago Ah, yes, sorry. I updated the Dockerfile recently to have a glibc based container, which has significant performance improvements. But that is of course not reflected in the latest tagged version.
See last two commits here: https://github.com/sando38/helm-ejabberd/commits/main/
So, either you patch it or you would need to rebuild the container image using this Dockerfile as a reference: https://github.com/sando38/helm-ejabberd/blob/ejabberd-0.8.0/image/24.02/Dockerfile
MTRNord
commented
6 months ago Seems like I have at least the elector error still with the tag. The nc bug is gone though. So I guess thats some progress:
2024-05-16 17:58:02.117620+00:00 [info] <0.186.0>@ejabberd_config:load/1:82 Loading configuration from /opt/ejabberd/conf/ejabberd.yml
tail: logs/ejabberd.log has appeared; following end of new file
tail: logs/error.log has appeared; following end of new file
2024-05-16 17:58:02.633764+00:00 [info] <0.186.0>@ejabberd_config:load/1:89 Configuration loaded successfully
2024-05-16 17:58:02.790133+00:00 [info] <0.431.0>@ejabberd_systemd:init/1:94 Got no NOTIFY_SOCKET, notifications disabled
2024-05-16 17:58:02.816235+00:00 [info] <0.434.0>@translate:load/2:127 Building language translation cache
2024-05-16 17:58:02.934601+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'ejabberd_commands'
2024-05-16 17:58:02.975792+00:00 [warning] <0.522.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 100
2024-05-16 17:58:02.975910+00:00 [warning] <0.522.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 5000
2024-05-16 17:58:02.975997+00:00 [warning] <0.522.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 100
2024-05-16 17:58:02.976069+00:00 [warning] <0.522.0>@ejabberd_shaper:resolve_shapers/3:233 Shaper rule 'max_user_offline_messages' refers to unknown shaper: 5000
2024-05-16 17:58:03.018723+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'route'
2024-05-16 17:58:03.027874+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'route_multicast'
2024-05-16 17:58:03.045849+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'session'
2024-05-16 17:58:03.049586+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'session_counter'
2024-05-16 17:58:03.060238+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 's2s'
2024-05-16 17:58:03.063346+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'temporarily_blocked'
2024-05-16 17:58:03.073818+00:00 [info] <0.430.0>@gen_mod:start_modules/0:141 Loading modules for xmpp.midnightthoughts.space
2024-05-16 17:58:03.886670+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'pubsub_last_item'
2024-05-16 17:58:03.957567+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'mod_register_ip'
2024-05-16 17:58:04.012862+00:00 [info] <0.674.0>@mod_stun_disco:parse_listener/1:626 Going to offer STUN service: 100.96.7.253:3478 (udp)
2024-05-16 17:58:04.108978+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'muc_online_room'
2024-05-16 17:58:04.306943+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'bosh'
2024-05-16 17:58:04.356266+00:00 [info] <0.381.0>@ejabberd_mnesia:create/2:270 Creating Mnesia ram table 'bytestream'
2024-05-16 17:58:04.365032+00:00 [warning] <0.731.0>@ejabberd_captcha:get_prog_name/0:444 The option captcha_cmd is not configured, but some module wants to use the CAPTCHA feature.
2024-05-16 17:58:04.438512+00:00 [info] <0.186.0>@ejabberd_cluster_mnesia:wait_for_sync/1:123 Waiting for Mnesia synchronization to complete
2024-05-16 17:58:04.521093+00:00 [info] <0.186.0>@ejabberd_app:start/2:63 ejabberd 24.2.0 is started in the node 'ejabberd@ejabberd-0.ejabberd-headless.ejabberd.svc.cluster.local' in 2.57s
2024-05-16 17:58:04.521490+00:00 [info] <0.517.0>@ejabberd_listener:init/4:129 Start accepting UDP connections at :3478 for ejabberd_stun
2024-05-16 17:58:04.521552+00:00 [info] <0.723.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at 100.96.7.253:7777 for mod_proxy65_stream
2024-05-16 17:58:04.521673+00:00 [info] <0.513.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at :5222 for ejabberd_c2s
2024-05-16 17:58:04.521710+00:00 [info] <0.514.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at :5280 for ejabberd_http
2024-05-16 17:58:04.521758+00:00 [info] <0.515.0>@ejabberd_listener:init/4:163 Start accepting TLS connections at :5443 for ejabberd_http
2024-05-16 17:58:04.521656+00:00 [info] <0.512.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at 127.0.0.1:5281 for ejabberd_http
2024-05-16 17:58:04.521799+00:00 [info] <0.516.0>@ejabberd_listener:init/4:163 Start accepting TCP connections at :5269 for ejabberd_s2s_in
run.sh: ==> Start elector sidecar service on 127.0.0.1:4040 ...
run.sh: ==> Wait for elector sidecar to be available on 127.0.0.1:4040 ...
/usr/local/bin/elector: line 0: syntax error: unexpected word (expecting ")")
sh: can't kill pid 246: No such processOk, I see. I just checked the underlying elector binary and its Makefile. It seems the command make build-linux in the Dockerfile only compiles an x86-64 variant. Can you try to exchange that with:
https://github.com/sando38/k8s-elector/blob/master/Makefile
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -installsuffix cgo -ldflags "${LDFLAGS}" -o elector cmd/elector.goThat did it! Now it is working :)
sando38
commented
6 months ago Thanks for the feedback!
Hi the Readme mentions that arm images are are currently not existing but "So there will be soon an arm64 variant as well." after that. How long of a "soon" can we expect? Is it more like weeks/a month or is it more like half a year?
Or other way to phrase it: Should I go ahead and rather build the image myself or is it reasonable to wait a bit on them instead? :) Mainly asking as I run an arm64-only cluster :)