Aggregation ReadStream intermittently fails to deliver results to the handlers

[gaganis]

Version

4.3.1

Context

We have a test in our testsuite that is testing an aggration query that we have developed we are running this aggregation with io.vertx.ext.mongo.MongoClient#aggregate. We occasionally have ci failures because our code hangs and does not finish.

Our investigations have led us to suspect that aggregation results are not delivered to the handler of the ReadStream we are using to process the results of the aggregation.

Additional Information

While debugging the problem we have seen by accident (we were trying to see if running resume like functions would make the stream deliverer the results) that running io.vertx.ext.mongo.impl.PublisherAdapter#requestMore on the readstrem object would trigger the ReadStream to deliver the results and the test proceeds and succeeds.

Do you have a reproducer?

This reproducer contains one RepeatedTest which when run in intellij there appear repetitions that fail:

mongodb-aggregation-reproducer.zip

Extra

• The reproducer has been developed on a MacBook Pro but we have originally seen the bug also in CI which is running Linux.

[tsegismont]

Thanks for the report, I'll give a try to the reproducer.

[tsegismont]

This happens because the version of the Mongo Java driver enforced by the Spring Boot BOM is not compatible with the Vert.x Mongo Client.

I don't understand yet what is the problem but the big difference is that the new versions uses Spring Reactor as its Reactive Streams implementation (it was not the case before).

So, currently, if you want to use Vert.x Mongo Client in a SB app, you have to force the compatible version:

<mongodb.version>4.1.2</mongodb.version>

      <dependency>
          <groupId>org.mongodb</groupId>
          <artifactId>bson</artifactId>
          <version>${mongodb.version}</version>
      </dependency>
      <dependency>
          <groupId>org.mongodb</groupId>
          <artifactId>mongodb-driver-core</artifactId>
          <version>${mongodb.version}</version>
      </dependency>
      <dependency>
          <groupId>org.mongodb</groupId>
          <artifactId>mongodb-driver-reactivestreams</artifactId>
          <version>${mongodb.version}</version>
      </dependency>

[gaganis]

Thanks for the investigation and providing the recommendation. We have tried it and it not working out for us.

It appears that Spring Boot(in our case 2.7.0) assumes a different version of the driver and is not compatible itself with your proposed version of the driver.

We see the following:

nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'mongoDatabaseFactory' defined in class path resource [org/springframework/boot/autoconfigure/data/mongo/MongoDatabaseFactoryConfiguration.class]: Unsatisfied dependency expressed through method 'mongoDatabaseFactory' parameter 0; 
nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mongo' defined in class path resource [org/springframework/boot/autoconfigure/mongo/MongoAutoConfiguration.class]: Bean instantiation via factory method failed;
nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.mongodb.client.MongoClient]: Factory method 'mongo' threw exception; 
nested exception is java.lang.NoClassDefFoundError: com/mongodb/internal/connection/InternalConnectionPoolSettings

Do you guys plan to provide a newer version of the Vert.x Mongo Client that is compatible with the latest version of the MongoDB driver?

[tsegismont]

It appears that Spring Boot(in our case 2.7.0) assumes a different version of the driver and is not compatible itself with your proposed version of the driver.

I have tried myself and it worked, the trick is to remove the dependencies on Spring Boot Mongo starter and embedded mongo. Do you really need them if your app uses Vert.x Mongo Client?

Do you guys plan to provide a newer version of the Vert.x Mongo Client that is compatible with the latest version of the MongoDB driver?

Yes we do, but not until at least the next minor release (the next one is a micro release, 4.3.2)

[gaganis]

Do you really need them if your app uses Vert.x Mongo Client?

Yes we need them, our application also has parts that use the Sprint Boot client that have not been migrated to use Vertx.

Yes we do, but not until at least the next minor release (the next one is a micro release, 4.3.2)

Do you have any new information on when it is planned to provide a compatible version?

[tsegismont]

Do you have any new information on when it is planned to provide a compatible version?

Not yet

[tsegismont]

@gaganis just wanted to let you know we're making some progress on #290

[tsegismont]

Closing, progress is tracked in #289