Pool of connections

[dnesteryuk]

Hello Florin,

Turns out I misunderstand something regarding the pool. When I call:

Bolt.Sips.conn

I expect to get an established connection to a Neo4j instance, but actually, it starts establishing a new connection from scratch, it doesn't get it from the pool. The Bolt.Sips.Connection uses the pool to be sure there is a limited number of process which establish connections, but those processes don't preserve established connections for further use. The connection operation is quite expensive (40-50 ms in my case).

Did I get it wrong? I am looking forward for your response. Thanks. :smile:

[florinpatrascu]

Nope, you got it right 😄

When you call Bolt.Sips.conn, there is indeed a new connection created, provided there are available slots (in the pool) - one of the ways, in Elixir, we limit the consumers, otherwise you can easily exhaust the server resources (and yours too) if you allow concurrent processes to run arbitrarily.

People are often asking me: "-Hey, in Java we have a session for stuff like this!" Not here, not needed (most of the time). If you want to reuse a conn (if it's expensive?) then simply reuse it! Same thing happens in Phoenix - via the Plug.Conn; you pass around the connection until you're done with it.

conn = Bolt.Sips.conn
my_awesome_method(conn)
my_very_cool_method(conn)
# and so on

For example, it is very important to reuse the Bolt.Sips connection, when planning to use transactions! Example:

test "commit statements in an open transaction" do
    try do
      conn = Bolt.Sips.begin(Bolt.Sips.conn)
      book = Bolt.Sips.query(conn, "CREATE (x:XactCommit {foo: 'bar'}) return x")
      assert {:ok, [row]} = book
      assert row["x"].properties["foo"] == "bar"

      # Main connection should not see this new node.
      {:ok, results} = Bolt.Sips.query(Bolt.Sips.conn, "MATCH (x:XactCommit) RETURN x")
      assert is_list(results)
      assert Enum.count(results) == 0,
          "Main connection should not be able to see transactional changes"

      # Now, commit...
      Bolt.Sips.commit(conn)

      # And we should see it now with the main connection.
      {:ok, [%{"x" => node}]} = Bolt.Sips.query(Bolt.Sips.conn, "MATCH (x:XactCommit) RETURN x")
      assert node.labels == ["XactCommit"]
      assert node.properties["foo"] == "bar"

    after
      # Delete any XactCommit nodes that were succesfully committed!
      Bolt.Sips.query(Bolt.Sips.conn, "MATCH (x:XactCommit) DETACH DELETE x")
    end
  end

Anyway, that's the story of it. Bolt.Sips is simple, as advertised.

I am open to accept PRs implementing different pools, if you find one where is happening what you need (see: fishcakez/connection, as a possible example) but even then you'll have to manage the pool in your own client code i.e.close(conn), Session.close(conn) and so on. And you'll still have to pass the connection/session/etc around, to reuse it, as you know.

HTH Florin