subscribe can timeout for no good reason

[brianmay]

Seems to be when making multiple subscribe requests, sometimes tortoise Tortoise.Connection.subscribe_sync function will timeout, and there doesn't appear to be any good reason for it doing so.

Retrying the failed function call doesn't help, it also times outs.

The server is operating fine in the meantime.

[brianmay]

I am now seeing this error regularly:

19:34:44.456 [error] GenServer {Tortoise.Registry, {Tortoise.Connection, "robotica-canidae"}} terminating
** (FunctionClauseError) no function clause matching in Tortoise.Connection.handle_info/2
    (tortoise 0.9.4) lib/tortoise/connection.ex:380: Tortoise.Connection.handle_info({{Tortoise, "robotica-canidae"}, #Reference<0.3137246812.2896691204.5918>, ["state/AnotherBrian/Light/power"]}, %Tortoise.Connection{backoff: %Tortoise.Connection.Backoff{max_interval: 30000, min_interval: 100, value: nil}, client_id: "robotica-canidae", connect: %Tortoise.Package.Connect{__META__: %Tortoise.Package.Meta{flags: 0, opcode: 1}, clean_session: false, client_id: "robotica-canidae", keep_alive: 60, password: "<censored>", protocol: "MQTT", protocol_version: 4, user_name: "brian", will: nil}, keep_alive: #Reference<0.3137246812.2896691202.5536>, opts: [client_id: "robotica-canidae", handler: {Robotica.Client, []}], server: %Tortoise.Transport{host: 'mqtt.linuxpenguins.xyz', opts: [:binary, {:packet, :raw}, {:active, false}, {:verify, :verify_peer}, {:cacertfile, "/etc/ssl/certs/ISRG_Root_X1.pem"}], port: 8883, type: Tortoise.Transport.SSL}, status: :up, subscriptions: %Tortoise.Package.Subscribe{__META__: %Tortoise.Package.Meta{flags: 2, opcode: 8}, identifier: nil, topics: [{"execute", 0}, {"mark", 0}, {"request/all/#", 0}, {"request/robotica-canidae/#", 0}, {"state/AnotherBrian/Light/power", 0}]}})
    (stdlib 3.12.1) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib 3.12.1) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib 3.12.1) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

[brianmay]

I think problem is that dynamic subscriptions/subscriptions just don't work reliably at present time :-(

[arjun289]

@brianmay I am facing a similar issue. Our use case requires us to create dynamic subscriptions when some new artifact is created. Sometimes the subscriptions don't handle messages, although the connection still exists. Sometimes the connection is lost abruptly. I am not able to reproduce this. Have you found any workaround to handle this?

[brianmay]

I ended up moving to an erlang library for mqtt:

https://github.com/emqx/emqtt

Which I have written a wrapper for connection management. i.e. if the connection goes does it will try to reconnect and then re-establish all dynamic subscriptions:

https://github.com/brianmay/mqtt_potion/

Only problem, is that while there is a hex repo, there isn't any interest in keeping the hex repo up-to-date:

https://github.com/emqx/emqtt/issues/133

Which in turn means I can't publish my library to hex either, because hex doesn't like dependencies to git repos.

It also wants to pull in quic dependencies by default. Which probably you don't want or need. Define the BUILD_WITHOUT_QUIC=true env variable to stop this. Haven't worked out how to get dependabot to define this yet though, so dependabot is constantly adding quic, and then I am removing it.

Documentation is sadly lacking also. Often it is a case of "use the source" to find answers to questions.

Regardless of the above issues, it does seem like a reasonable quality library, and this has resolved all the issues I have had with the tortoise project.

While the idea behind tortoise mqtt5 rewrite may have been good (see the mqtt5 branch), unfortunately efforts have completely stalled.

Also see https://elixirforum.com/t/any-recommendations-on-a-mqtt-client-for-elixir/41028

Regarding my mqtt_potion library, have also considered porting my mqtt subscription service to this library also:

https://github.com/brianmay/robotica-elixir/blob/master/robotica/lib/robotica/subscriptions.ex

This allows different processes to subscribe to specific topics over a single mqtt connection, and will automatically unsubscribe when the processes die. e.g. useful for Pheonix live views. Probably needs some documentation also.. Support for wildcard topics would also be good.

Hope this helps :-)

[arjun289]

@brianmay thanks for the detailed description. I will try this out. I hope this keeps your connections stable.