is there any documentation about the functions available in ejabberd?

[salmanasiddiqui]

I am trying to develop a custom module in elixir, in which I need to access the configured database (creating a new table and storing data in it on certain event). The list of events is not available in new documentation but i was able to find it in the old docs, but even in old docs its not defined when those events are fired. From where can I get the list of existing modules, functions and records in ejabberd which I can use to access the database?

[zinid]

There is no such documentation. If you want simple put/get/delete operations you can look at the code of mod_private_*.erl modules. More generally, if you want to use SQL, you can use functions from ejabberd_sq.erl module. As an example of usage you can check the code of any mod_X_sql.erl modules.

[zinid]

Also, if you have particular questions, like "I want to read value X, do something with it and then write value Y", you can ask here.

[salmanasiddiqui]

I have the data that I want to write to my custom table but I am unable to.

On user_send_packet I invoke a function which invokes the function below:

store_xml(#message{id = Id, from = From, to = To, body = Body} = Msg) ->
  LServer = From#jid.lserver,
  DBHost = gen_mod:get_module_opt(LServer, ?MODULE, db_host, LServer),
  LogQuery = fun() ->
        ejabberd_sql:sql_query_t(
          "INSERT INTO custom_table(raw_packet, unique_id, message, from_jid, to_jid) VALUES(" ++ Msg ++ ", "
            ++ binary_to_list(Id) ++  ", " ++ binary_to_list(xmpp:get_text(Body)) ++ ", " ++ format_jid(From) ++ ", " ++ format_jid(To) ++ ");"
        )
      end,
  ejabberd_sql:sql_transaction(DBHost, LogQuery),
  ok.

But nothing is inserted in the DB. There is no error in ejabberd logs.

EDIT: On printing the result of sql_transaction i got the following error: [{aborted,{badarg,[{erlang, ++ .... so the issue is in concatenating Msg to rest of the string. Can you tell me how can I convert this Msg object to string?

[zinid]

There are several problems with your code:

• You probably don't need to use transaction here. Transactions are heavy, don't use them if you don't need atomicity. Use ejabberd_sql:sql_query() instead.
• You are messing ++ operator with different types. The operator only accepts lists, however, Msg is of type message() and I'm not sure about the return type of format_jid/1 function (it should return a list() in your case).

how can I convert this Msg object to string?

You should encode it to xmlel() first (that is, Erlang representation of XML) using xmpp:encode() function (see the manual or API reference of the xmpp library) then you should serialize it (i.e. converting to binary()) using fxml:element_to_binary() funtion. Finally, you should convert it to list() in order to apply ++ operator using list_to_binary/1. So, it should look something like that:

binary_to_list(fxml:element_to_binary(xmpp:encode(Msg)))

[salmanasiddiqui]

thanks @zinid , even after doing the above I ran into issues of escaping the binary_to_list(fxml:element_to_binary(xmpp:encode(Msg)))

So then I used the ?SQL macro, and I had to remove the binary_to_list conversion for that to work. ?SQL macro wasn't working at first, but then I added -compile([{parse_transform, ejabberd_sql_pt}]) after which it started to work. I am unable to find documentation about what -compile does in erlang, can you share some link where I can find this info?

Also whats the recommended way of creating a table for my custom ejabberd module? I have created the table directly for now but that doesn't make it reusable, is there anyway to put .sql in some directory which would be read by ejabberd on startup?

[zinid]

I am unable to find documentation about what -compile does in erlang, can you share some link where I can find this info?

Options of function compile:file/2 can be used in -compile() directive

is there anyway to put .sql in some directory which would be read by ejabberd on startup?

There is nothing like this in ejabberd, but on our TODO list.

[salmanasiddiqui]

Alright, thanks again for your help @zinid . I have just 2 more questions now,

1. Is ejabberd_redis recommended to interact with redis? or custom module should use eredis directly? I see that list related functions of redis are not implemented in ejabberd_redis so I doubt its recommended to be used.

2. If I implement my own blocking module, how can I stop a packet from reaching the receiver? Incase of exceptions the msg is still delivered, whats the way to halt the callback chain? is it by returning xmpp:err_* from our custom functions hooked to events?

EDIT: Also, is the list of events in the old documentation outdated? is the signature of user_send_packet correct in that documentation?

[zinid]

1. All redis requests should be performed via ejabberd_redis only, because it maintains a pool of connections. You can add new function to this module and propose a PR.
2. You can always return stop from the hooks without an accumulator (i.e. run-hook) and {stop, Acc} from the hooks with accumulator (i.e. run_fold-hook). See the guide. Note that this will only stop further execution of a list of hooked functions. If you want to just drop a message, you should do it in some hooks called inside ejabberd_c2s.erl, such as user_send_packet, user_receive_packet of privacy_check_packet.

[salmanasiddiqui]

ok, I see ejabberd_redis doesnt take host as an argument like ejabberd_sql does for its functions. It also doesn't support module level config, I want to connect to separate redis server/db, and push something to that redis from my custom module. I do not think I can make use of ejabberd_redis module, may be I should write up my own custom_redis module to maintain its own pool of connections and use that in my custom module? What would you recommend @zinid ?

[zinid]

Well, in such case you have to write your module, yes.

[salmanasiddiqui]

@zinid ,

Question

how can I start same server multiple times?

Context

I have created one ejabberd module mod_test, it contains two files:

1. mod_test.erl(which is of gen_mod behavior) mod_test.erl is a simple module, which on start hooks a function to offline_message_hook and on stop it deletes that hook.

2. custom_redis.erl (which is of gen_server behaviour) custom_redis.erl is kind of stripped down version of ejabberd_redis.erl

Now when my module mod_test starts I want to start custom_redis.erl multiple times (depending on the pool size). I see from the documentation that we can call gen_mod:start_child.. So I tried to do the following but it didnt work:

-module(mod_test).
-behavior(gen_mod).

start(Host, Opts) ->
  ...
  gen_mod:start_child(custom_redis, Host, 1),
  gen_mod:start_child(custom_redis, Host, 2),
  ok.

but it seems that the first command starts a child, but second command doesn't do anything.

On a separate note

the function I added to offline_message_hook was not being executed because it had different parameters. I was finding the expected parameters from Event and hooks page, but it has outdated signatures for hooks.

offline_message_hook(From, To, Packet) -> ok

While current signature is offline_message_hook(Action, Packet)

[zinid]

Two many questions, please ask a single question at a time, it's hard to address them.

[zinid]

You cannot use functions from gen_mod for something that is not of gen_mod behaviour. So you should create your own supervisor and attach your custom_redis processes to it.

[zinid]

Regarding offline: I didn't understand the question. There are a lot of functions attached to the hook and they work.

[salmanasiddiqui]

@zinid , I have updated my last comment to make it more clear. There is just one question about running multiple processes of custom_redis(gen_server) from mod_test(gen_mod). At the end of my comment, I just wanted to let you know that the documentation page on Events and Hooks has outdated info.

[salmanasiddiqui]

@zinid in user_send_packet hook I returned {stop, {drop, C2SState}}. And this seems to drop the packet without spitting any exception in the log. Can you confirm this is the right way to do it?

At first I returned stop and then {stop, {Packet, C2SState}} but it wasn't stopping the message to be sent to the user. Then I returned drop and {drop, {Packet, C2SState}} but both of them were spitting some errors in the log.

[zinid]

Can you confirm this is the right way to do it?

Yes, if the intention is to drop the message silently. That's an absolutely correct way doing this.

[salmanasiddiqui]

Thank you, How can I send a packet to someone from my custom module?

Scenario: UserA sends a msg to UserB, on user_send_packet before returning Acc. I want to send delivery receipt to UserA that the msg is delivered.

[zinid]

So send it in the same hook, using ejabberd_router:route/1.

[zinid]

Or you can send it using ejabberd_c2s:send(State, Msg).

[salmanasiddiqui]

thanks again @zinid ,

Using swift.im or adium works fine with ejabberd and my custom modules. I am trying to replace tigase with ejabberd, I connected my current clients (android app using Smack) with ejabberd and I got the following error:

2017-09-22 09:56:02.947 [debug] <0.17033.0>@ejabberd_receiver:process_data:284 Received XML on stream = <<"<?xml version=\"1.0\" encoding=\"UTF-8\"?><message to=\"be061579d5cfaff7d7a6f19bdb2556f84524b73b@somehost.com\" from=\"02d343f08a021312d77623cfff5d7c906d07f944@somehost.com\" type=\"chat\" id=\"387c4c4b-40f4-43bc-84f5-72fb5fdc24c8\"><read unique_id=\"\">1506074160</read></message><?xml version=\"1.0\" encoding=\"UTF-8\"?><message to=\"be061579d5cfaff7d7a6f19bdb2556f84524b73b@somehost.com\" from=\"02d343f08a021312d77623cfff5d7c906d07f944@somehost.com\" type=\"chat\" id=\"7596689e-8849-47ab-8a2f-9c683219709e\"><read unique_id=\"\">1506074163</read></message>">>

2017-09-22 09:56:02.947 [debug] <0.17033.0>@shaper:update:143 State: {maxrate,1000,53.070507960576194,1506074162554167}, Size=694 M=356.458722741433, I=393.206

2017-09-22 09:56:02.947 [debug] <0.17034.0>@ejabberd_socket:send:216 (tcp|<0.17033.0>) Send XML on stream = <<"<stream:error><not-well-formed xmlns='urn:ietf:params:xml:ns:xmpp-streams'/><text xml:lang='en' xmlns='urn:ietf:params:xml:ns:xmpp-streams'>XML or text declaration not at start of entity</text></stream:error>">>

Received xml from above log:

<<"
  <?xml version=\"1.0\" encoding=\"UTF-8\"?>
  <message to=\"be061579d5cfaff7d7a6f19bdb2556f84524b73b@somehost.com\" from=\"02d343f08a021312d77623cfff5d7c906d07f944@somehost.com\" type=\"chat\" id=\"387c4c4b-40f4-43bc-84f5-72fb5fdc24c8\">
    <read unique_id=\"\">1506074160</read>
  </message>
  <?xml version=\"1.0\" encoding=\"UTF-8\"?>
  <message to=\"be061579d5cfaff7d7a6f19bdb2556f84524b73b@somehost.com\" from=\"02d343f08a021312d77623cfff5d7c906d07f944@somehost.com\" type=\"chat\" id=\"7596689e-8849-47ab-8a2f-9c683219709e\">
    <read unique_id=\"\">1506074163</read>
  </message>
">>

I can't figure anything wrong with the packet sent by the client, plus such packets are working fine with tigase.

[zinid]

You cannot include declarations inside an already opened stream, ejabberd actually tells you this:

XML or text declaration not at start of entity

[salmanasiddiqui]

thanks.. after removing declaration from every packet, chat started to work fine.

To connect to an external mysql database, is there any other way to achieve it except for creating my own custom_sql.erl and custom_sql_sup.erl. Will I need to create custom_sql_pt.erl and custom_sql_pt.hrl aswell ?

I do not completely understand the parse_transformof -compile yet. And why these sql_pt.erl and sql_pt.hrl does.

[zinid]

It depends. If you want to rewrite SQL code significantly, then you will have to cope with parse transformers (which is quite brutal). You probably don't need to use your own SQL handlers, what is lacking for you in standard ones?

[salmanasiddiqui]

Well I have configured mysql as default dB for ejabberd, I have added few tables to it and created modules which use those tables. Now I need to connect to other mysql server and run some queries there.

As per my current knowledge of Erlang and ejabberd, that is not possible currently and I will have to write my own custom_sql and custom_sql_pt.. but this parse transformer thing is quite complicated so I just wanted to be sure if I need to learn that or I can just get it done by custom_sql and custom_sql_pt.

I am hoping that I will be using the ?SQL macro but instead of calling ejabberd_sql:sql_query() I will call custom_sql:sql_query(). Is that doable without parse transformers?

[cromain]

i'm closing this thread, which contains so much different points, and is out of the scope of this project.

[Zedonboy]

Please @zinid can just give me an example of elixir database query with :ejabberd_sql.

[zinid]

@Zedonboy

$ grep 'sql_query' ejabberd/src/*.erl | wc -l
336

So 336 examples in ejabberd code is not enough for you? You need yet another one?

[Zedonboy]

@zinid dont be offended sir, its just they are written in Erlang. i see something like this. ejabberd:sql_query(Lserver, ?SQL("select ...")). i really dont know how to do that in elixir.

[zinid]

Ah, Elixir. Well, regarding Elixir, I already tired to explain: if you choose Elixir to avoid reading/learning Erlang then you are wrong. As wrong as those guys choosing Scala to avoid Java. So eventually you'll end up in reading/writing both languages, which is not bad by itself but was definitely not the purpose people chose Elixir for.