ActiveMQ and JMS broker - [merged] #146

Closed jdonkervliet closed 4 years ago

In GitLab by @larsdetombe on May 19, 2020, 15:22

Merges feature/activemq-broker -> development

Belongs to issues #11 #12

This merge request adds a JMS broker on which ActiveMQ can run.

In GitLab by @JimVliet on May 19, 2020, 20:10

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 56

Furthermore, we should avoid redundant copying here too. We should investigate if it is possible to directly write the data to bytesMessage. The buffer.array() method could work, although I already noticed that some of the ByteBuf implementations do not support this operation. Therefore, it might be worth considering ByteBuf implementations that do support this.

In GitLab by @swabbur on May 19, 2020, 20:16

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 56

Correct, it would be empty up to index. However, the index should always be 0 here and if it is not, it means that something else must have been written to and read from the same buffer before (which we then probably shouldn't read).

As for the second comment, the use of buffer.array() is reserved for array-based ByteBuf implementations. To use such a buffer, we would need to be able to separately read the length before reading the message, or be able to determine the maximum length of all messages (which is near impossible). I agree that we should look into skipping the additional transfer of data. To do this, we need to be able to read the length of the message before fully reading it, or by determining what the maximum message size is.

In GitLab by @JimVliet on May 19, 2020, 20:17

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsSubscriber.java line 15

If the consumer is garbage collected the callback will not be garbage collected as long as another reference to it exists. If that is not the case, then why are we keeping track of it anyway?

In GitLab by @JimVliet on May 19, 2020, 20:18

Was it actually tested in practice?

In GitLab by @swabbur on May 19, 2020, 20:18

We haven't tried it yet. Though it is hard to say, since we haven't fully integrated the messaging system yet and still need to add the benchmarking dependency we were planning to use.

In GitLab by @swabbur on May 19, 2020, 20:24

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsSubscriber.java line 15

Hmmm... That's true. JMS must be keeping track of the listeners/subscribers somewhere anyways.

In GitLab by @larsdetombe on May 19, 2020, 20:36

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsBroker.java line 125

It does not require a lot of additional code, so the computeIfAbsent is indeed better

In GitLab by @larsdetombe on May 19, 2020, 20:38

I did run the activeMQ broker, but it is very hard to say what performs better. It is hard for me to spot the difference with the naked eye.

In GitLab by @larsdetombe on May 19, 2020, 20:42

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsBroker.java line 135

The problem that I had with that, is that I would have to create a JmsSubscriber with a null consumer to use the contains method. Then afterward we would have to create another JmsSubscriber with the actual consumer.

In GitLab by @JimVliet on May 19, 2020, 20:50

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 56

The documentation of the getBytes is ambiguous:

Transfers this buffer's data to the specified destination starting at the specified absolute index. This method does not modify readerIndex or writerIndex of this buffer.

I initially thought they meant that the data will be written to the bytes array starting at specified index. This would result in the first few bytes of the bytes array all being zero bytes.

After looking through the netty code I found an example implemenation:

    @Override
    public CompositeByteBuf getBytes(int index, byte[] dst) {
        return getBytes(index, dst, 0, dst.length);
    }

As you can see the 0 is actually the destination index. So the first byte read will be written to the start of the bytes array.

Basically, I thought the index indicated the position where the writing would start in the bytes array. Instead it is of course the position where the reading will start.

Apologies, for my confusion.

In GitLab by @JimVliet on May 19, 2020, 20:59

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 56

Regarding the second point, I later found this SO post that confirms what you said. Currently, I do not have any idea on how to remove this copying going on here. The main problem seems to be the fact that bytesMessage.writeBytes(...) only accepts byte arrays and not the ByteBuf of netty. It is possible of course to write directly using a simple for loop with a single writeByte call, but writing single bytes at a time is most likely even more inefficient than copying the data to the byte array.

In GitLab by @JimVliet on May 19, 2020, 21:06

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsBroker.java line 30

I do not think this is a major issue and if you all disagree, feel free to resolve this.

In GitLab by @JimVliet on May 19, 2020, 21:07

I suppose that is a good indication. We will probably only know more once we start benchmarking the implementations.

In GitLab by @JimVliet on May 19, 2020, 21:11

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsBroker.java line 135

The problem with using a stream is that you have to iterate through the set, instead of using the hashcode, which is more efficient.

In GitLab by @swabbur on May 20, 2020, 08:42

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 56

Writing a single byte at a time would indeed remove the need for the additional buffer. But, I agree that it won't perform much better (if not worse).

We could reduce the cost of using the additional buffer by using a pool. Instead of continuously allocating new ones, we should reuse the ones we have already created.

In GitLab by @swabbur on May 20, 2020, 08:47

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsBroker.java line 135

Completely forgot about that, gotten so used to all the additional methods on Kotlin's data structures...

The overhead of allocating an additional object (especially such a simple one, which will probably be optimized away by the compiler) is negligible compared to having to iterate over all the objects in the collection.

In GitLab by @swabbur on May 20, 2020, 08:48

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 36

Could you use a full name here? (registration instead of reg as it could be confused with registry)

In GitLab by @swabbur on May 20, 2020, 08:50

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 46

Could you split this line into two separate ones:

Codec codec = registration.getCodec();
messageBuf = codec.encode(messageBuf, message);

And, does the encode method already store the result in the messageBuf? If so, the return value could be ignored.

In GitLab by @swabbur on May 20, 2020, 08:54

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 73

Indeed, having the length there is a habit from working in C (where "arrays" do not know their own length, and not defining it could lead to a buffer overflow).

In GitLab by @JimVliet on May 20, 2020, 10:19

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 46

In my opinion this is a bit overkill, since it is just a simple get method.

In GitLab by @larsdetombe on May 20, 2020, 13:55

Commented on src/main/java/net/glowstone/messaging/brokers/jms/serializers/ProtocolCodec.java line 46

I personally prefer to it this way as well, so ill change it.

In GitLab by @larsdetombe on May 20, 2020, 14:07

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsBroker.java line 158

changed this line in version 4 of the diff

In GitLab by @larsdetombe on May 20, 2020, 14:07

Commented on src/main/java/net/glowstone/messaging/brokers/jms/JmsBroker.java line 125

changed this line in version 4 of the diff

In GitLab by @larsdetombe on May 20, 2020, 14:07

Commented on src/main/java/net/glowstone/messaging/brokers/Brokers.java line 23

changed this line in version 4 of the diff

In GitLab by @larsdetombe on May 20, 2020, 14:07

Commented on src/main/java/net/glowstone/messaging/brokers/Brokers.java line 38

changed this line in version 4 of the diff