Persistent connection

[stefanak-michal]

Take a look at this. https://github.com/neo4j-php/Bolt/pull/115

https://www.php.net/manual/en/function.pfsockopen.php

[stefanak-michal]

• maybe use https://www.php.net/manual/en/ref.sem.php for shared memory instead of files to keep serverState per connection
• check https://www.php.net/manual/en/function.stream-get-meta-data.php if can be used to get uid of connection for storage

[stefanak-michal]

You can even use stream_socket_client (used in StreamSocket connection class) with flag STREAM_CLIENT_PERSISTENT to start using persistent connections.

But what is hard to do (maybe impossible) is to tell in what state is that connection. Neo4j doesn't offer bolt message to request server state or some kind of status. Or even I don't know if there is a way how to ask if that connection was closed. Because after I called goodbye bolt message I get that persistent connection again.

To tell which connection is which is possible with stream_socket_get_name($this->stream, false) .. I think, because you get local outbound port.

[stefanak-michal]

Added new branch with bold version which does work. When you create connection, call hello (do some query if you want) and then refresh second time without calling hello again. Because the connection is already logged in. So I used this kind of condition to verify it:

if ($protocol->serverState->get() == \Bolt\protocol\ServerState::CONNECTED)

For now I keep socket connection info in file. Also I still need to manage how to recover. For example when persistent connection doesnt exists but there is still that file. It needs more work and testing.

I used bolt v5 which means hello logs in. In newer version was changed to logon after hello.

[stefanak-michal]

What about using https://www.php.net/manual/en/function.apcu-store.php for storing handlers for persistent connections?

[transistive]

I like the idea, but the problem is that it needs to be installed on your PHP environment. I think we should use something like a PSR cache instead. People can then install their own implementation, which may or may not use apcu, redis, etc ...

https://www.php-fig.org/psr/psr-16/

[stefanak-michal]

I spent few more hours on this feature. I've implemented PSR-16 and did some clean up. But there are few problems I can see or I hit.

1. When you lost metadata for connection - system will try handshake and it will fail on timeout when waiting for response.
2. Not consuming response immediately - if you keep pipelining messages and then you refresh script, you will not be able to consume responses in second script run. When you send message, type of message is buffered and this buffer is used for consuming. Reloading script lost this buffer.
3. I even end up in state when I had to restart apache while I was experimenting with leaving connection in different Server states and trying to reconnect and do something. Reusing existing connection properly heavily depends on correct action related to Server state.

About the 2. point. Maybe I should look into it if I can remove that buffer somehow. I feel like the buffer is there just to tell Response what message is related to. But I don't think is used anywhere. Is it required to know it?

[stefanak-michal]

I just realized one problem: Because of pipeline, response from one script run can end up in another.

So it would make sense to now have pipeline at all as it was before and consume response immediatelly after sending any message to database. Which feels like step back because pipeline allows consuming result continuously (better memory usage).

[stefanak-michal]

I was thinking, calling reset upon connecting with persistent connection would solve some problems.

[stefanak-michal]

I was thinking, calling reset upon connecting with persistent connection would solve some problems.

Reset doesn't discard buffer, just mark all messages as IGNORED. So I've implemented discarding all bytes from buffer upon reusing persistent connection.

Also it's called RESET to assure connection is viable (SUCCESS response).

So it seems I've finalized this feature. It is implemented as new connection class PStreamSocket.

I'm not able to solve one problem which is when meta information get lost. Because of that upon connection it tries to do handshake (because it thinks it's a new connection) which fails and it throws ConnectionTimeoutException. I mentioned that in class annotation.

What I haven't tested: When there is persistent connection but is not used for longer time, server can close it. I don't know what exactly happens when you try to reuse it. My guess is, PHP internally close it too. But if PHP creates new connection while meta information still exists, I guess library throws ConnectionException on write/read.