IndexOutOfBoundsException in Sender

[DerekSchenk]

I'm attempting to use send messages between two instances, both using Vertx-proton, one side initiates connections and the other side responds. I'm using defaults for all the connection options. I suspect my issue has to do with how I'm using Vertx-proton, but I can't seem to determine what I'm doing wrong.

If I send messages with a delay between them everything works, but if I remove the delay I'll receive an IndexOutOfBoundsException. The exception always happens at various locations inside FrameWriterBuffer while writing the frame.

This is the code I'm using to send (stripped down to test). For this test, the receiving side just drops the message.

    for(int x=0; x<100; x++) {
            List<Data> batch = new ArrayList<>();
            batch.add(new Data(new Binary(new byte[40])));
            Section section = new AmqpSequence(batch);
            Message m = Proton.message();
            m.setBody(section);
            _sender.send(m);
        }

If I add a handler with a latch (see below) I can stop the send side error.

                        CountDownLatch latch = new CountDownLatch(1);
                        _sender.send(m, e->{
                            latch.countDown();
                        });
                        latch.await();

However, if I have the other side respond to messages (same logic as above) on the same link I start getting the IndexOutOfBoundsException's on both sides.

[ERROR] [ForkJoinPool-1-worker-9:ca.datex.nexus.core.communication.vertx.ChannelVertx$OutboundConsumer:178] [2019-06-27 08:02:29.792] java.lang.IndexOutOfBoundsException: readerIndex: 156, writerIndex: 78 (expected: 0 <= readerIndex <= writerIndex <= capacity(1024)) at io.netty.buffer.AbstractByteBuf.checkIndexBounds(AbstractByteBuf.java:111) at io.netty.buffer.AbstractByteBuf.readerIndex(AbstractByteBuf.java:120) at io.vertx.proton.impl.ProtonReadableBufferImpl.position(ProtonReadableBufferImpl.java:156) at org.apache.qpid.proton.engine.impl.FrameWriterBuffer.put(FrameWriterBuffer.java:134) at org.apache.qpid.proton.engine.impl.FrameWriter.writeFrame(FrameWriter.java:114) at org.apache.qpid.proton.engine.impl.TransportImpl.writeFrame(TransportImpl.java:1125) at org.apache.qpid.proton.engine.impl.TransportImpl.processTransportWorkSender(TransportImpl.java:636) at org.apache.qpid.proton.engine.impl.TransportImpl.processTransportWork(TransportImpl.java:535) at org.apache.qpid.proton.engine.impl.TransportImpl.writeInto(TransportImpl.java:363) at org.apache.qpid.proton.engine.impl.TransportOutputAdaptor.pending(TransportOutputAdaptor.java:59) at org.apache.qpid.proton.engine.impl.SaslImpl$SwitchingSaslTransportWrapper.pending(SaslImpl.java:842) at org.apache.qpid.proton.engine.impl.HandshakeSniffingTransportWrapper.pending(HandshakeSniffingTransportWrapper.java:138) at org.apache.qpid.proton.engine.impl.TransportImpl.pending(TransportImpl.java:1577) at org.apache.qpid.proton.engine.impl.TransportImpl.getOutputBuffer(TransportImpl.java:1526) at io.vertx.proton.impl.ProtonTransport.flush(ProtonTransport.java:259) at io.vertx.proton.impl.ProtonConnectionImpl.flush(ProtonConnectionImpl.java:296) at io.vertx.proton.impl.ProtonSenderImpl.send(ProtonSenderImpl.java:106) at io.vertx.proton.impl.ProtonSenderImpl.send(ProtonSenderImpl.java:56)

Any assistance or guidance on the correct way to use the library would be appreciated.

[gemmellr]

Your note of latching making a difference perhaps points to timing from threading issues, and it looks like the thread name is "ForkJoinPool-1-worker-". Are you using things outwith the vertx event loop context thread? If so, that will be the problem, and isnt supported as its expressly single threaded, e.g see: https://github.com/vert-x3/vertx-proton/blob/3.5/src/main/asciidoc/java/index.adoc#threading-considerations

Beyond that, the snippets of code you give dont really give enough detail to see whats going on without more context.

As more of an aside, is there a reason you are embedding Data body sections inside an AmqpSequence section? Embedding one type of body section inside another like that is a bit unexpected, and you could just use Binary elements without the Data section 'wrapping', doesnt really add anything at that point. Less jarring or important, you could perhaps also use a List inside an AmqpValue section rather than using an AmqpSequence section.

[DerekSchenk]

Thanks for the quick response. I'm just trying to get a fully standalone version of the code to reproduce the issue.

Part of the logic I'm trying to implement involves batching messages when more than 1 exists, which is why I was also using a sequence. The use of the Data inside the Sequence was leftover from testing and I removed it however it didn't address the problem.

There is a queue on each side that holds messages pending transmission, and I use a single thread to read from the queue, batch if needed, and then transmit. I tried using the Context to execute the send method however that led to strange results where some messages get sent repeatedly and others not at all.

[gemmellr]

If you are using it from outwith the event loop thread then no need for reproducer, that is certainly going to be the issue and you need to stop doing that. It will always cause issues such as this obvious one or other less obvious ones.

Removing the odd use of Data wont affect anything in regard to the problem, it was just an aside that its strange to do that since Data is a body section, and so not something you typically put inside another body.

[DerekSchenk]

I used the feedback above, and also found the project vertx-amqp-bridge which is close to the functionality I was looking for, and I modified my logic to use the event bus as well as using a process similar to the bridge to move the messages between instances. This resolved the exception as well as the repeated messaages. Thanks for the assistance.

[gemmellr]

From your comment I dont think you used the bridge, just looked at it, but just in case and more so for anyone else reading later...

The amqp-bridge (which unlike some of the other bridges doesn't use the event bus, just implements some of its APIs) will be removed in Vert.x 4 and so I would not suggest it be used for new work, use either the newer https://github.com/vert-x3/vertx-amqp-client, or vertx-proton (upon which both others are built).