Stable connection

[zethuman]

Greetings to those who read!

TLDR

I would like to discuss or raise the issue with a stable connection. I have the following situation, at the moment I'm trying to write a production ready service that could provide more than 50 connections per second on one instance. But, during the tests, I encountered a not very good problem, which now puts me in an awkward position. If the format of any field is incorrect, when the client receives a safe error, my connection is closed. Next, I expected some kind of connection attempt, during the echo test, but did not wait. I began to dig and see if there is such a mechanism, but unfortunately I did not find it ...

Summary

I have three questions:

1. How to restore the connection when receiving a fatal error or incorrect response from the server?
2. How to restore the connection when the server closes the connection, let's say it restarted or there was a short-term connection loss?
3. Do I need a pool of connections in my case, or is it possible to get by with one connection?

Glad for any answer or discussion

UPDATE

Here is a results of my debugging, as you can see pack error is not recognized as PackError, so it breaks and closes the connection. Is it a correct behaviour?

Solution

Wrap error as PackError to bypass connection close. I think it is not obvious, so you should debug or read all source code of lib to find out.

rawMessage, err := message.Pack()
if err != nil {
      return utils.NewSafeError(&connection.PackError{Err: err}, "failed to pack message")
}

[alovak]

Hey @zethuman ! thanks for submitting the issue!

Wanted to clarify this one:

I have the following situation, at the moment I'm trying to write a production-ready service that could provide more than 50 connections per second on one instance.

Do you mean you want to process 50 messages per second? I'm asking as usually there is a few connections that are established with the payment network (2, 4, 10) and such connections are (in some sense) permanent - you use the connection to send and receive many messages and you don't close it.

In this test, you can see how we concurrently send and receive 10 messages on the same connection.

If the connection was closed (by the server or system) - there is no way to restore it. You should create a new connection. connection.Pool does exactly this - creates a new connection when the old one is closed. Please, check this test to see that when one server is down, Pool is trying to create and establish a new connection. I hope that this paragraph is the answer to all your 3 questions.

For the PackError. We have a test here that proves (as it seems) that the code you mentioned should work. From your screenshot I'm not seeing that the error wraps PackError and that's strange. I'll check what can be wrong here. Do you set a custom MessageWriter?

[zethuman]

Hi, @alovak ! Glad to see you again

Do you mean you want to process 50 messages per second?

Yeap, while developing, I aim for a value of 100 requests per second, since this is a limitation of the channel itself, so I wanted my service not to fail. Even if I hit 50 per second, I can lift two instances at once to cope with the load. But I would like to achieve as large a value in seconds as possible.

connection.Pool does exactly this

Yes, I just have a choice on the pool or on single connection. If the server somehow falls off, I can restart my service until the service rises. Or I can use a connection pool, but I thought about the appropriateness of this approach, since from the descriptions I thought that a connection pool is used when there are several servers, but I have only one.

Do you set a custom MessageWriter?

Yes, I have a custom one, because the service specification needs to be played with a hex before packing and unpacking. Therefore, I noted that it is not obvious that you need to wrap the error in a specific format, since this was not written in the documentation, to understand this, you had to debug or read the source code.

[alovak]

I think maybe we should return PackError from iso8583, then there will be no difference if you use default message reader or your own.

For the Pool - you can use it with one server and one connection. This will work faster than restarting whole service.

For the 50 messages/seconds - there should be no issues with such load even on one machine and connection, but for reliability you would need at least two servers that process messages.

[alovak]

@zethuman I prepared the PR https://github.com/moov-io/iso8583/pull/249. When it's merged we can update the iso8583-connection to handle this error types and then custom message reader and writer don't have to worry about returning error types.

[zethuman]

@alovak thanks for covering all my questions

[alovak]

@zethuman this PR will solve the issue as you don't have to wrap errors yourself anymore.