Closed gloffreda closed 3 years ago
ioolkos
commented
3 years ago Can you try sudo apt install libsnappy-dev.
I'm not sure why the rebar3 command works though.
(JFYI a note on the compiled packages and commercial use: they require a paid subscription as per the EULA)
gloffreda
commented
3 years ago Thanks for your prompt reply.
I did a revision, you hit a good point, as I'm using a VM to make that tests it was virgin and missing some libs.
I'm aware about EULA, in our production environment we use v1.9.2 and we are running some tests with 1.10.1 and now 1.11 compiling it from source. I've been talking with Andre some months ago. Our point is a support subscription that makes sense for our size and demand. At that time, there wasn't any. Is it still the case ?
Thanks again
ioolkos
commented
3 years ago apologies, usually I keep marketing out of Github issues :)
Jep, I guess you're missing libsnappy dependency. Also build-essentials need to be on the system, but you probably have that.
gloffreda
commented
3 years ago I could make it happen but now I'm getting an execution error on my container:
/vernemq/bin/../erts-10.6.4/bin/cuttlefish: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
Thanks
ioolkos
commented
3 years ago So you're missing that library. Haven't checked but I guess this is one of those installed by build-essential.
gloffreda
commented
3 years ago that's weird
I have the package build-essential.
On my VM where i'm compiling the sources it states no errors.
Then I build my docker image using a tar file generated from _build/default/rel/vernemq folder.
Only when I start the docker with start_vernemq I get the error:
/vernemq/bin/../erts-10.6.4/bin/cuttlefish: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
Which seems the lib is missing on my image, not a problem with compiled application, am I seem correctly ? Then I installed build-essentials and libtinfo-dev on my image. Still having the issue.
any tip welcome
ioolkos
commented
3 years ago I think I have never seen that specific error, and the only issue on this is https://github.com/vernemq/vernemq/issues/1704 (which is for Mac OS X) In the Docker build, do you actually do those "apt install"'s (libsnapp and build-essential)? Yeah, it's not an issue with the compiled app, it's a linking issue it seems.
gloffreda
commented
3 years ago Yes, I tried to use the Dockerfile used on official packages to not reinvent the wheels. There, we don't have the build-essential neither. Even though I tried to install it on apt-get. Same result. Even last version on github of docker gives exact same error. is there something new on 1.11 ?
FROM debian:stretch-slim
RUN apt-get update && \
apt-get -y install bash procps openssl iproute2 curl jq && \
rm -rf /var/lib/apt/lists/* && \
addgroup --gid 10000 vernemq && \
adduser --uid 10000 --system --ingroup vernemq --home /vernemq --disabled-password vernemq
WORKDIR /vernemq
# Defaults
ENV DOCKER_VERNEMQ_KUBERNETES_LABEL_SELECTOR="app=vernemq" \
DOCKER_VERNEMQ_LOG__CONSOLE=console \
PATH="/vernemq/bin:$PATH" \
VERNEMQ_VERSION="1.11.0"
COPY --chown=10000:10000 bin/vernemq.sh /usr/sbin/start_vernemq
COPY --chown=10000:10000 files/vm.args /vernemq/etc/vm.args
COPY versions/vernemq-$VERNEMQ_VERSION.stretch.tar.gz /tmp
RUN tar -xzvf /tmp/vernemq-$VERNEMQ_VERSION.stretch.tar.gz && \
rm /tmp/vernemq-$VERNEMQ_VERSION.stretch.tar.gz && \
chown -R 10000:10000 /vernemq && \
ln -s /vernemq/etc /etc/vernemq && \
ln -s /vernemq/data /var/lib/vernemq && \
ln -s /vernemq/log /var/log/vernemq
# Ports
# 1883 MQTT
# 8883 MQTT/SSL
# 8080 MQTT WebSockets
# 44053 VerneMQ Message Distribution
# 4369 EPMD - Erlang Port Mapper Daemon
# 8888 Prometheus Metrics
# 9100 9101 9102 9103 9104 9105 9106 9107 9108 9109 Specific Distributed Erlang Port Range
EXPOSE 1883 8883 8080 44053 4369 8888 \
9100 9101 9102 9103 9104 9105 9106 9107 9108 9109
VOLUME ["/vernemq/log", "/vernemq/data", "/vernemq/etc"]
COPY mongodb.lua /vernemq/share/lua/auth/mongodb.lua
#COPY vernemq.conf /etc/vernemq/vernemq.conf
HEALTHCHECK CMD vernemq ping | grep -q pong
USER vernemq
CMD ["start_vernemq"]One thing maybe can help.
On downloaded tar file, we have erts-10.7 folder.
On my compiled package I have erts-10.6.4.
What should I do to update to 10.7 ?
gloffreda
commented
3 years ago I manage to make it. I've compiled using ubuntu 20.04. When I run my docker it was using Debian Stretch. Some libs used on ubuntu don't exists on Stretch. OTP and Erlang packages for ubuntu and debian are different. Then it crashes when starts on a new OS. I handled it by changing my docker to use ubuntu 20.04.
Just by curiosity, I would like to ask, when you generate the different images you use different OS to generate it ? Thank you for your help
gloffreda
commented
3 years ago Ok. Need yet a little help. After setting up, all went good. Then I turned diversity for mongodb on and a bunch of logs started to show up.
My mongodb credentials and host are all good. Tested with official image and it goes good.
Is there some tweak to make on diversity module ? I notice one thing. To compile the app, I'm using a VM with ubuntu. Does it need to have same directory structure that will run on docker image ? I notice some logs with my VM path.
3/2/2021 8:13:59 PMconfig is OK
3/2/2021 8:13:59 PM-config /vernemq/data/generated.configs/app.2021.03.02.23.13.57.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args
3/2/2021 8:13:59 PMExec: /vernemq/bin/../erts-11.1.7/bin/erlexec -boot /vernemq/bin/../releases/1.11.0+build.33.refd5a0ef9/vernemq -config /vernemq/data/generated.configs/app.2021.03.02.23.13.57.config -args_file /vernemq/bin/../etc/vm.args -vm_args /vernemq/bin/../etc/vm.args -pa /vernemq/bin/../lib/erlio-patches -- console -noshell -noinput
3/2/2021 8:13:59 PMRoot: /vernemq/bin/..
3/2/2021 8:13:59 PM23:13:59.555 [info] alarm_handler: {set,{system_memory_high_watermark,[]}}
3/2/2021 8:13:59 PM23:13:59.675 [info] writing state {[{[{actor,<<148,125,28,142,218,232,85,92,205,40,25,255,163,99,32,174,18,3,23,187>>}],1}],{dict,1,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[['VerneMQ@10.42.136.247',{[{actor,<<148,125,28,142,218,232,85,92,205,40,25,255,163,99,32,174,18,3,23,187>>}],1}]],[],[],[],[],[],[],[],[]}}},{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}} to disk <<75,2,131,80,0,0,0,251,120,1,203,96,206,97,96,96,96,204,96,130,82,41,12,172,137,201,37,249,69,185,64,81,145,41,181,50,125,183,94,132,198,156,213,144,252,191,56,89,97,157,16,179,248,238,172,68,198,172,12,206,20,6,150,148,204,228,146,68,198,68,1,32,228,72,12,72,52,200,16,200,66,3,25,140,112,49,176,233,32,130,41,133,65,52,44,181,40,47,213,55,208,193,208,64,207,196,72,207,208,216,76,207,200,196,156,52,23,192,109,66,56,133,129,72,167,192,181,2,0,125,202,76,124>>
3/2/2021 8:13:59 PM23:13:59.691 [info] Datadir ./data/meta/meta/0 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1482163}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.701 [info] Datadir ./data/meta/meta/1 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1875232}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.711 [info] Datadir ./data/meta/meta/2 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1824637}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.719 [info] Datadir ./data/meta/meta/3 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1890397}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.729 [info] Datadir ./data/meta/meta/4 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1301643}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.740 [info] Datadir ./data/meta/meta/5 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1302441}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.750 [info] Datadir ./data/meta/meta/6 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1857925}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.760 [info] Datadir ./data/meta/meta/7 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1580451}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.769 [info] Datadir ./data/meta/meta/8 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1512631}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.778 [info] Datadir ./data/meta/meta/9 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1367760}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.787 [info] Datadir ./data/meta/meta/10 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1927196}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.797 [info] Datadir ./data/meta/meta/11 options for LevelDB: [{open,[{block_cache_threshold,33554432},{block_restart_interval,16},{block_size_steps,16},{compression,true},{create_if_missing,true},{delete_threshold,1000},{eleveldb_threads,71},{fadvise_willneed,false},{limited_developer_mem,false},{sst_block_size,4096},{tiered_slow_level,0},{total_leveldb_mem_percent,6},{use_bloomfilter,true},{write_buffer_size,1992024}]},{read,[{verify_checksums,true}]},{write,[{sync,false}]},{fold,[{verify_checksums,true},{fill_cache,false}]}]
3/2/2021 8:13:59 PM23:13:59.902 [info] Try to start vmq_plumtree: ok
3/2/2021 8:13:59 PM23:13:59.965 [info] Opening LevelDB database at "./data/msgstore/1"
3/2/2021 8:13:59 PM23:13:59.981 [info] Opening LevelDB database at "./data/msgstore/2"
3/2/2021 8:13:59 PM23:13:59.994 [info] Opening LevelDB database at "./data/msgstore/3"
3/2/2021 8:14:00 PM23:14:00.007 [info] Opening LevelDB database at "./data/msgstore/4"
3/2/2021 8:14:00 PM23:14:00.020 [info] Opening LevelDB database at "./data/msgstore/5"
3/2/2021 8:14:00 PM23:14:00.034 [info] Opening LevelDB database at "./data/msgstore/6"
3/2/2021 8:14:00 PM23:14:00.046 [info] Opening LevelDB database at "./data/msgstore/7"
3/2/2021 8:14:00 PM23:14:00.058 [info] Opening LevelDB database at "./data/msgstore/8"
3/2/2021 8:14:00 PM23:14:00.070 [info] Opening LevelDB database at "./data/msgstore/9"
3/2/2021 8:14:00 PM23:14:00.088 [info] Opening LevelDB database at "./data/msgstore/10"
3/2/2021 8:14:00 PM23:14:00.103 [info] Opening LevelDB database at "./data/msgstore/11"
3/2/2021 8:14:00 PM23:14:00.120 [info] Opening LevelDB database at "./data/msgstore/12"
3/2/2021 8:14:00 PM23:14:00.213 [info] Try to start vmq_generic_msg_store: ok
3/2/2021 8:14:00 PM23:14:00.473 [info] loaded 0 subscriptions into vmq_reg_trie
3/2/2021 8:14:00 PM23:14:00.480 [info] cluster event handler 'vmq_cluster' registered
3/2/2021 8:14:01 PM __ ____ __ ____ _____ _______ ________ _____ _____ _____ _________ __
3/2/2021 8:14:01 PM \ \ / / \/ |/ __ \ | __ \_ _\ \ / / ____| __ \ / ____|_ _|__ __\ \ / /
3/2/2021 8:14:01 PM \ \ / /| \ / | | | |______| | | || | \ \ / /| |__ | |__) | (___ | | | | \ \_/ /
3/2/2021 8:14:01 PM \ \/ / | |\/| | | | |______| | | || | \ \/ / | __| | _ / \___ \ | | | | \ /
3/2/2021 8:14:01 PM \ / | | | | |__| | | |__| || |_ \ / | |____| | \ \ ____) |_| |_ | | | |
3/2/2021 8:14:01 PM \/ |_| |_|\___\_\ |_____/_____| \/ |______|_| \_\_____/|_____| |_| |_|
3/2/2021 8:14:01 PM
3/2/2021 8:14:01 PM23:14:01.284 [info] enable auth script for mongodb "./share/lua/auth/mongodb.lua"
3/2/2021 8:14:01 PM23:14:01.495 [error] gen_server <0.473.0> terminated with reason: <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55
3/2/2021 8:14:01 PM23:14:01.495 [error] CRASH REPORT Process <0.473.0> with 1 neighbours crashed with reason: <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55
3/2/2021 8:14:01 PM23:14:01.495 [error] Supervisor {<0.470.0>,poolboy_sup} had child vmq_diversity_worker_wrapper started with {vmq_diversity_worker_wrapper,start_link,undefined} at <0.473.0> exit with reason <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55 in context child_terminated
3/2/2021 8:14:01 PM23:14:01.496 [error] gen_server <0.471.0> terminated with reason: <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55
3/2/2021 8:14:01 PM23:14:01.496 [error] CRASH REPORT Process <0.471.0> with 1 neighbours crashed with reason: <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55
3/2/2021 8:14:01 PM23:14:01.497 [error] Supervisor {<0.470.0>,poolboy_sup} had child vmq_diversity_worker_wrapper started with {vmq_diversity_worker_wrapper,start_link,undefined} at <0.471.0> exit with reason <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55 in context child_terminated
3/2/2021 8:14:01 PM23:14:01.497 [error] gen_server <0.475.0> terminated with reason: <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55
3/2/2021 8:14:01 PM23:14:01.497 [error] CRASH REPORT Process <0.475.0> with 1 neighbours crashed with reason: <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55
3/2/2021 8:14:01 PM23:14:01.497 [error] Supervisor {<0.470.0>,poolboy_sup} had child vmq_diversity_worker_wrapper started with {vmq_diversity_worker_wrapper,start_link,undefined} at <0.475.0> exit with reason <<"Can't pass authentification">> in mc_auth_logic:scram_sha_1_auth/4 line 55 in context child_terminated
varnerac
commented
3 years ago I think MongoDB auth was broken in 1.11. It works in the main branch from git.
Environment
I'm trying to compile from source the version 1.11 to learn how to tweak the auth modules. I've downloaded the zip, unpacked it and ran "make rel".
On both versions I'm getting same error. This is my output:
If I run
rebar3 releaseI get no erros:Any hint how to overcome it ? Thanks