MySQL Linking Issue with MongooseIM reference #1026

[vickydhas]

MongooseIM version: 2.0.0 (beta2) as latest code on git hub http://github.com/esl/MongooseIM. Installed from: source with reference to issue with fixings #1024 Erlang/OTP version: (put the version) :

Describe the issue.

MySql is not able to replace Mnesia when configuration changed and mongooseim restarted.

Steps done : ...mypathof compiled source.../etc/ejabberd.cfg changes made to make sure MySQL data base is used.

1. used the /..../MongooseIM/apps/ejabberd/priv/mysql.sql file to create schema using SQL workbench
2. Tables created as seen in the schema.
3. Log Level to 5 for more information @ ##87 {loglevel, 5}.
4. Made the MySQL Database settings in below lines

%% %% MySQL server: %% {odbc_server, {mysql, "localhost", 3306, "ejabberd", "root", "abcd@123"}}. %%

1. %% Authentication : I see that the stack guys asking to make the 310 line uncomment if using another database than mnesia and so If I make the 218 commented and 310 used I get the above issue as reported, else everything works fine but mysql linking is not happening.

   281 {auth_method, internal }.

   310 %%{auth_method, odbc}.

2. restarted mongooseIM by mongooseim stop & start call
3. Added some dummy users using mongooseimctl register user host password command
4. checked mysql table is filled or not no data present for newly added user.

Am i missing something ???

[michalwski]

If you want to have your users in MySQL you should use {auth_method, odbc}. More info about configuring MongooseIM with external DB is here [1] and here [2].

If you still have this issue, it'd be helpful if you could provide server logs from file ejabberd.log and the entire ejabberd.cfg (cleaned form sensitive data)

[1]. http://mongooseim.readthedocs.io/en/latest/advanced-configuration/database-backends-configuration/ [2]. http://mongooseim.readthedocs.io/en/latest/Advanced-configuration/#database-setup

[vickydhas]

Ive done this {auth_method, odbc} and commented the {auth_method, internal} previous when i restarted the mongoose it gave me error but now there is no logs in the log directoy.

[vickydhas]

Very weird problem happening , I can now make the build and start the server but the ctl commands just seems not responding. Nor i can connect any users on chat.

ejabberd.docx ejabberd.docx

sh-3.2$ bin/mongooseim start mkdir: log: Permission denied sh-3.2$ sudo bin/mongooseim start Node is already running! sh-3.2$ sudo bin/mongooseimctl status MongooseIM node mongooseim@localhost: operating system pid: 15681 Erlang VM status: starting (of: starting | started | stopping) boot script status: applications_loaded version: 2.0.0beta2-284-g2786b03 uptime: 0 days 00:00:20 logs: none - maybe enable logging to a file in app.config? sh-3.2$ sudo bin/mongooseim ping pong sh-3.2$ sudo bin/mongooseimctl registered_users localhost Failed RPC connection to the node mongooseim@localhost: {'EXIT', {badarg, [{ets,lookup, [local_config, mongooseimctl_access_commands], []}, {ejabberd_config, get_local_option, 1, [{file, "src/ejabberd_config.erl"}, {line,749}]}, {ejabberd_ctl, get_accesscommands, 0, [{file, "src/ejabberd_ctl.erl"}, {line,248}]}, {ejabberd_ctl, process,1, [{file, "src/ejabberd_ctl.erl"}, {line,209}]}, {rpc, '-handle_call_call/6-fun-0-', 5, [{file,"rpc.erl"}, {line,187}]}]}}

Attached my ejabberd.cfg file. Also note that I have not seen any log files in MongooseIM/rel/mongooseim/logs as its empty.

[michalwski]

This smells like an issues with file permissions because of sudo you used to build the project. I suggest stopping all Erlang process by killall beam.smp

Now, you have 2 options:

1. Start from scratch, but please, do not use sudo this time, never :)
2. Use sudo, but always.

I personally recommend the first option.

[michalwski]

Regarding the config file you uploaded, could you please upload them again but in original format?

[vickydhas]

Okay when I clone out I don't have the mongooseIM folder onlynits there after I build.

Okay I will follow your instructions to not use sudo. Will do it and pass it to you.

Thanks again.

On Thursday, 13 October 2016, Michał Piotrowski notifications@github.com wrote:

Regarding the config file you uploaded, could you please upload them again but in original format?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/esl/MongooseIM/issues/1031#issuecomment-253485321, or mute the thread https://github.com/notifications/unsubscribe-auth/AAy8Z1FImE36q1MkCMgkal5IVOHWaYUJks5qzhNDgaJpZM4KVnv5 .

Sent from my iPhone6+

[vickydhas]

ERLANG VERSION : Erlang/OTP 19 [erts-8.1] [source-77fb4f8] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] Eshell V8.1 (abort with ^G)

MIM VERSION: MongooseIM version 2.0.0beta2-284-g2786b03

Hello Michal,

This time I have done as mentioned by you.

1) Checkout source code to /users/username/projects/MongooseIM 2) No Su command used anywhere 3) Navigated to '/users/username/projects/MongooseIM' root directory and entered commange as given below

LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make rel

4) All build was successfully done except last warning for (cannot run generate command on my current folder)

5) mongooseim folder created with all folders and executables in the 'rel' folder.

6) Navigated to rel/mongooseim folder and ran following command

$bin/mingooseim start  (Server was started successfully)

$bin/mingooseim ping (Server replied 'pong' on the next line) $ bin/mongooseimctl registered_users localhost AND $ bin/mongooseimctl register test1 localhost test1

=== OR ===

sh-3.2$ bin/mongooseimctl registered_users localhost AND sh-3.2$ bin/mongooseimctl register test1 localhost test1 ```

7) In the 6th last commands I got this error, where I have neither changed any permission or used sudo command.

`Failed RPC connection to the node mongooseim@localhost: {'EXIT',
                                                         {badarg,
                                                          [{ets,lookup,
                                                            [local_config,
                                                             mongooseimctl_access_commands],
                                                            []},
                                                           {ejabberd_config,
                                                            get_local_option,
                                                            1,
                                                            [{file,
                                                              "src/ejabberd_config.erl"},
                                                             {line,749}]},
                                                           {ejabberd_ctl,
                                                            get_accesscommands,
                                                            0,
                                                            [{file,
                                                              "src/ejabberd_ctl.erl"},
                                                             {line,248}]},
                                                           {ejabberd_ctl,
                                                            process,1,
                                                            [{file,
                                                              "src/ejabberd_ctl.erl"},
                                                             {line,209}]},
                                                           {rpc,
                                                            '-handle_call_call/6-fun-0-',
                                                            5,
                                                            [{file,"rpc.erl"},
                                                             {line,187}]}]}}`

8) There is no file in the log directory, where will I see this `ejabberd.log` file in this source code as I didn't find it in `rel/logs`folder of current source code compiled./

[vickydhas]

I've attached the text files for your reference, log files from log directory.

crash.txt ejabberd.txt

[michalwski]

Thanks for the attached files. From the crash.txt I can see, that Erlang was not compiled with crypto. Most probably you didn't have openssl installed while you were installing Erlang. How did you install Erlang?

[vickydhas]

I installed using command $brew install erlang

On Monday, 17 October 2016, Michał Piotrowski notifications@github.com wrote:

Thanks for the attached files. From the crash.txt I can see, that Erlang was not compiled with crypto. Most probably you didn't have openssl installed while you were installing Erlang. How did you install Erlang?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/esl/MongooseIM/issues/1031#issuecomment-254121348, or mute the thread https://github.com/notifications/unsubscribe-auth/AAy8Z-cdZg1pPcsw9M_TOUr5H3EWrpERks5q0xMrgaJpZM4KVnv5 .

Sent from my iPhone6+

[michalwski]

Do you know if you installed it before or after installing openssl?

[vickydhas]

Today once again i installed the erlang to start from fresh, opensll was already installed from that.

[vickydhas]

Ok I will download erlang OTP source code and compile with openssl and get back to you.

[michalwski]

The easiest would probably be to use kerl (https://github.com/kerl/kerl) and following config in your ~/.kerlrc file:

KERL_CONFIGURE_OPTIONS="--disable-hipe --enable-smp-support --enable-threads \
            --enable-kernel-poll --enable-darwin-64bit \
            --with-ssl=/usr/local/opt/openssl"

This works for me but I have Mac OS X El Capitan still.

[vickydhas]

Ok installed kerl and getting this at command prompt.

Vickys-MBP:bin vickydhas$ kerl

kerl: build and install Erlang/OTP usage: /usr/local/bin/kerl [options ...]

Command to be executed

Valid commands are: build Build specified release or git repository install Install the specified release at the given location deploy Deploy the specified installation to the given host and location update Update the list of available releases from your source provider list List releases, builds and installations delete Delete builds and installations active Print the path of the active installation plt Print Dialyzer PLT path for the active installation status Print available builds and installations prompt Print a string suitable for insertion in prompt cleanup Remove compilation artifacts (use after installation) version Print current version (current: 1.3.4)

Can you direct now what to do ?

[michalwski]

I can direct you to kerl's doc: https://github.com/kerl/kerl/blob/master/README.md :) It's described there how to build and install a specific version of Erlang. I'm not able to come up with more detailed hints.

[vickydhas]

I tried to use the kerl with many releases but getting this error in one of the latest build , I think jsut getting deviated from problem we started :)

Machines-MBP:~ /mymachine/$ kerl build R15B 19.1 Downloading otp_src_R15B.tar.gz to /Users//mymachine//.kerl/archives % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 178 100 178 0 0 160 0 0:00:01 0:00:01 --:--:-- 160 100 70.6M 100 70.6M 0 0 176k 0 0:06:50 0:06:50 --:--:-- 181k Getting checksum file from erlang.org... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 178 100 178 0 0 232 0 --:--:-- --:--:-- --:--:-- 232 100 25931 100 25931 0 0 7808 0 0:00:03 0:00:03 --:--:-- 17141 Verifying archive checksum... Checksum verified (dd6c2a4807551b4a8a536067bde31d73) Extracting source code Building Erlang/OTP R15B (19.1), please wait... APPLICATIONS INFORMATION (See: /Users//mymachine//.kerl/builds/19.1/otp_build_R15B.log)

• wx : Can not combine 64bits erlang with wxWidgets on MacOSX, wx will not be useable

DOCUMENTATION INFORMATION (See: /Users//mymachine//.kerl/builds/19.1/otp_build_R15B.log)

• documentation :
• fop is missing.
• Using fakefop to generate placeholder PDF files.

Build failed. gcc -m64 -g -O2 -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_FCNTL_H=1 -DHAVE_NETDB_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_SYS_SOCKET_H=1 -DHAVE_STRUCT_SOCKADDR_IN6_SIN6_ADDR=1 -DHAVE_MEMSET=1 -DHAVE_SOCKET=1 -DHAVE_PTHREAD_H=1 -I. -I/usr/lib/include -I/Users//mymachine//.kerl/builds/19.1/otp_src_R15B/lib/erl_interface/include -DUNIX -o ../priv/obj/i386-apple-darwin16.0.0/odbcserver.o -c odbcserver.c odbcserver.c:117:10: fatal error: 'sql.h' file not found

include "sql.h"

     ^

1 error generated. make[4]: * [../priv/obj/i386-apple-darwin16.0.0/odbcserver.o] Error 1 make[3]: * [opt] Error 2 make[2]: * [opt] Error 2 make[1]: * [opt] Error 2 make: *\ [libs] Error 2

[michalwski]

Few questions:

1. What do you want to achieve by this command: kerl build R15B 19.1? Both R15B and 19.1 are Erlang versions. The naming schema changed a bit between R16 and 17. Current version of MongooseIM doesn't work with Erlang older then 17.
2. Did you put the kerl config to your ~/.kerlrc as I suggested?

[vickydhas]

1) My simple requirement is to generate a prototype for the chat app in XMPP, MongooseIM coz its more reliable and used previously. AWS hosting coz all the LAMP + MIM + DB will be stored on AWS. My SQL Chosen coz after prototype we don't have challenges to replace mnesia with MySQL or MicrosoftSQL as time limitations.

2) Yes I have put the kerl config in this file at path ~/.kerlrc

[michalwski]

The sql.h file are used to compile Erlang's ODBC lib, which you don't need. I suggest disabling it by adding KERL_CONFIGURE_DISABLE_APPLICATIONS="odbc" to the ~/.kerlrc file. More info about this here: https://github.com/kerl/kerl#tuning And in order to build Erlang/OTP 19 try this command: kerl build 19.1 19.1?

[vickydhas]

Okay I was able to add the KERL_CONFIGURE_DISABLE_APPLICATIONS as mentioned as well as able to install without any error the kerl build 19.1 19.1?

Now freshly downloaded the MIM and build using the LDFLAGS command and also the make rel, now I can get the $bin/mongooseim start command working but no ping or mongooseimctl commands working.

attached my log files for your reference.

[vickydhas]

Okay I was able to do the above to make a build uing Kerl 19.1 19.1 and also added the KERL_CONFIGURE_DISABLE_APPLICATIONS as asked in ~/.ker.. file

Installed fresh MongooseIM clone and build with LDFLags , the mongooseim start command works but not the ping and mongooseimctl now. Attached my log files from rel/mon.../logs folder.

Archive.zip

[michalwski]

Okay I was able to add the KERL_CONFIGURE_DISABLE_APPLICATIONS as mentioned as well as able to install without any error the kerl build 19.1 19.1

That's cool. Did you have any chance to actually install the build by command similar to the following one:

kerl install 19.1 ~/erlang/19.1

Documented in kerl's README.

Also you have to active this version by a command like this source /path/to/erlang/active, f.e

source ~/erlang/19.1/activate

I also recommend:

1. Uninstalling the erlang installed from brew by brew uninstall erlang
2. Adding path to the erlang installed from kerl to your PATH env var so that you don't have to activate it for every session.

After you do this, please rebuild MongooseIM so you are sure, the correct Erlang was used.

[vickydhas]

Thanks for your valuable advices and handholding for this Michal. I will try this today and come back.

And thanks to get me through each process as it's great fun to learn new things here, may be due to curiosity and limitation of knowledge on server side I may ask more questions.

Appreciate for your patience.

Sent from my iPhone6+

[vickydhas]

Hello Michal,

Below is the error I get now on sudo LDFlagmongoose rel

1) LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make Builds the mongoose im with no error

Output

./rebar get-deps > deps.log 2>&1 || (cat deps.log; exit 1)
./rebar  compile > compile.log 2>&1 || (cat compile.log; exit 1)

2) LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make rel gives error as mentioned.

ERROR: generate failed while processing /Users/vickydhas/mytemp/MongooseIM/rel: {'EXIT',{{badmatch,{error,"odbc: Missing application directory."}},
         [{rebar_reltool,generate,2,
                         [{file,"src/rebar_reltool.erl"},{line,53}]},
          {rebar_core,run_modules,4,[{file,"src/rebar_core.erl"},{line,493}]},
          {rebar_core,execute,6,[{file,"src/rebar_core.erl"},{line,418}]},
          {rebar_core,maybe_execute,8,
                      [{file,"src/rebar_core.erl"},{line,302}]},
          {rebar_core,process_dir1,7,[{file,"src/rebar_core.erl"},{line,261}]},
          {rebar_core,process_each,5,[{file,"src/rebar_core.erl"},{line,351}]},
          {rebar_core,process_dir1,7,[{file,"src/rebar_core.erl"},{line,253}]},
          {rebar_core,process_commands,2,
                      [{file,"src/rebar_core.erl"},{line,93}]}]}}
make: *** [rel] Error 1

As guided instructions I was able to install the kerl and also activate and build completely with documentation link and your steps.

3) Erlang version was checked on installation with $erl -version command and out put was.

Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 8.1

4) Kerl was specifically installed for me to test in custom path directory

Vickys-MBP:MongooseIM vickydhas$ kerl active

The current active installation is:
/Users/mymachine/erlang/19.1_build

I think something minor is remaining, also just to inform you I was successfully able to install the mongoose IM on ubuntu machine with few fixes and libraries installation.

Just wanted to check if this can be achieved on mac. As source code compilation may be faster as compared to work on VM on mac for ubuntu.

[michalwski]

It's probably last issue. I recommend:

1. make clean
2. remove file configure.out by rm configure.out
3. generate the release with make rel remember about LDFLAGS and CFLAGS

also just to inform you I was successfully able to install the mongoose IM on ubuntu machine with few fixes and libraries installation.

This is cool, I'm glad the whole process was smoother for you on Ubuntu.

[vickydhas]

Followed your steps and getting this issue. 1) make clean 👍 2) rm configure.out 👍 3) make & make rel using ID Flags O/p 👍

Vickys-MBP:MongooseIM vickydhas$ LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make

./rebar get-deps > deps.log 2>&1 || (cat deps.log; exit 1)
./rebar  compile > compile.log 2>&1 || (cat compile.log; exit 1)

Vickys-MBP:MongooseIM vickydhas$ LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make rel

./rebar get-deps > deps.log 2>&1 || (cat deps.log; exit 1)
./tools/configure with-all
....come more logs

==> riak_pb (compile)
==> rel (generate)
WARN:  'generate' command does not apply to directory /Users/vickydhas/mytemp/MongooseIM
Vickys-MBP:MongooseIM vickydhas$ sudo rel/mongooseim/bin/mongooseim start
Password:

Vickys-MBP:MongooseIM vickydhas$ sudo rel/mongooseim/bin/mongooseim ping
pong 👍 
Vickys-MBP:mongooseim vickydhas$ bin/mongooseimctl start
Node is already running! 👍 
Vickys-MBP:mongooseim vickydhas$ bin/mongooseimctl status 👍 
MongooseIM node mongooseim@localhost:
    operating system pid: 8557
    Erlang VM status: starting (of: starting | started | stopping)
    boot script status: applications_loaded
    version: 2.0.0beta2-284-g2786b03
    uptime: 0 days 00:00:12
    logs: none - maybe enable logging to a file in app.config?

Vickys-MBP:mongooseim vickydhas$ bin/mongooseimctl 👎  **Expected to show all ctl commands but show below error**

Failed RPC connection to the node mongooseim@localhost: nodedown
Commands to start a MongooseIM node:
  start  Start a MongooseIM node in server mode
  debug  Attach an interactive Erlang shell to a running MongooseIM node
  live   Start MongooseIM node in live (interactive) mode
MongooseIM cluster management commads:
  join_cluster other_node_name                Add current node to cluster
  leave_cluster                               Leave current node from the cluster
  remove_from_cluster other_node_name         Remove dead node from the cluster

Even if tried first without SUDO as you say or with Sudo the output remains same.

[michalwski]

Ok, let's do the following:

1. rm -rf rel/mongooseim
2. LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make rel
3. rel/mongooseim/bin/mongooseim start - please note there is NO sudo. You don't need it to start the server.
4. rel/mongooseim/bin/mongooseimctl

What's the result now?

[vickydhas]

This output only,

Failed RPC connection to the node mongooseim@localhost: nodedown. Logs-commands execution.txt

[michalwski]

Anything inside rel/mongooseim/logs?

[fenek]

@vickydhas

Can you please try rel/mongooseim/bin/mongooseim live instead of start? It will start the server in interactive mode with all errors printed in the console.

[vickydhas]

I have attached the console logs I received for the live mentioned by @fenek , @michalwski there is no logs directory generated on this command in rel folder.

mongooseim live logs.txt

I to give outputs with these commands executed always:

1) Clean : $make clean 2) Build : $sudo LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make. 3) Release : $sudo LDFLAGS=-L/usr/local/opt/openssl/lib CFLAGS=-I/usr/local/opt/openssl/include make rel.

[fenek]

It seems that using sudo caused problems with file permissions. Please execute chown -R youruser:yourgroup on repo directory and try live again.

[vickydhas]

Ok did as you expected. Attached the commands logs and erl_crash dump file.

erl_crash.dump.txt mongooseim live logs_1.txt

[vickydhas]

Hello All,

I think spent too much efforts on Mac machine to setup development environment but prefer to move to ubuntu for ease of application purpose.

With all this inputs I was able to successfully install and operate the MongooseIM server including the MySQL linkage as per docs.

I would be switching to virtual machine and ubuntu desktop running MIM stack over it and do necessary development.

Just to let you know so we close the case and resolve as workable with no issues on Ubuntu setup.

Secondly only problem I faced with ubuntu was to run some dependency like libssl, libexpat1,expat-Dev to be installed via sudo apt-get install commands.

Another important problem with cloning of latest master branch code is the tar files mostly the x86_64 ... gcov-tool.1.tar.gz in /usr/local/lib/..../man1.. directly comes corrupted and hence sudo make fails.

To resolve this I used option to find the file in by doing following commands

$cd /usr

$find -name "x86_64 ... gco*

I get some other paths than erlang so I use following command

$rm -r x86_64 ... gco file that's broken link in erlang directory, see the exact file error and check that file in erlang path.

$ln -s "other local usr directory path" "erlang directory path"

This will link the file in correctly

Then just sudo make clean, sudo make and $sudo make rel will be enough to make the 'rel' folder.

Later I just copied this rel directory and stored in my live public server and configured ejabberd.cfg file for domain and all settings.

Even the mysql issue got solved and I can see by data coming in MySQL rather than Mnesia.

Hope this helps someone who facing same issue.

@michal : if for curiosity we can continue to fix this but you may still close this issue from my side.

Appreciate for all guidance given as it helped a lot to setup the ubuntu setup.

Cheers. Vicky