Aggregation criteria match mapping fails with `NullPointerException`

[piotrmucha]

spring-data-mongodb version: 4.2.4

I am trying to move following query from JSON to MongoTemplate:

{
  '$match': {
      $expr: {
        $regexMatch: {
          input: {
            $toString:
              "fieldToConvert",
          },
          regex: "aa",
          options: "i",
        }
      }
  }
}

I have tried following approach:

var toString = ConvertOperators.valueOf("fieldToConvert").convertToString();
var regexMatch = StringOperators.valueOf(toString).regexMatch("aa", "i");
var expr = Criteria.expr(regexMatch);
var match = Aggregation.match(expr);
var aggregation = Aggregation.newAggregation(match);
mongoTemplate.aggregate(aggregation, "collectionName", classInstance);

But it doesn't work, exception like this is thrown:

Cannot invoke "org.springframework.data.mongodb.core.mapping.MongoPersistentEntity.getType()" because "entity" is null
    at org.springframework.data.mongodb.core.convert.QueryMapper.convertSimpleOrDocument(QueryMapper.java:592) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.convert.QueryMapper.getMappedKeyword(QueryMapper.java:411) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.convert.QueryMapper.getMappedObject(QueryMapper.java:130) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.aggregation.TypeBasedAggregationOperationContext.getMappedObject(TypeBasedAggregationOperationContext.java:82) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.aggregation.TypeBasedAggregationOperationContext.getMappedObject(TypeBasedAggregationOperationContext.java:77) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.aggregation.MatchOperation.toDocument(MatchOperation.java:74) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.aggregation.AggregationOperation.toPipelineStages(AggregationOperation.java:55) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.aggregation.AggregationOperationRenderer.toDocument(AggregationOperationRenderer.java:56) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.aggregation.AggregationPipeline.toDocuments(AggregationPipeline.java:86) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.aggregation.Aggregation.toPipeline(Aggregation.java:757) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.AggregationUtil.createPipeline(AggregationUtil.java:98) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2173) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2148) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2142) ~[spring-data-mongodb-4.2.4.jar:4.2.4]
    at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2021) ~[spring-data-mongodb-4.2.4.jar:4.2.4]

What's wrong here? Is it possible to use MongoExpression Criteria inside MatchOperation stage?

[christophstrobl]

@piotrmucha thanks for reporting - I need to check if what happens in the case of an untyped aggregation. It's likely the error stems from there.

[piotrmucha]

Thank you. I can confirm that replacing untyped aggregation with typed aggregation resolves the problem. So changing from:

var aggregation = Aggregation.newAggregation(match);

to:

var aggregation = Aggregation.newAggregation(classInstance, match);

Makes everyting works.

[christophstrobl]

thanks for checking @piotrmucha - we'll have a look to get it working with untyped ones as well.

[initdch]

I've encountered an issue similar to what @piotrmucha described, where the MatchOperation in an aggregation fails with certain _id formats. The test only passes when I modify the _id by adding a prefix.

Here's a concise summary of the problem:

I'm experienced an issue where a aggregation involving the MatchOperation fails when using certain _id formats. The test passes when I modify the _id by adding a prefix to it.

When attempting to match documents by _id using the MatchOperation, the operation fails if the _id is "66014bb53e3e9474cc0f39d2". However, modifying this ID by prefixing it with a character or a number (e.g., "A66014bb53e3e9474cc0f39d2") allows the test to pass. Both IDs are stored as strings in MongoDB.

@DataMongoTest
@Testcontainers
class TestIdBugRepositoryImplIntTest {

    private static final String ENTITY_ID_1 = "66014bb53e3e9474cc0f39d2";
    private static final String ENTITY_ID_2 = "166014bb53e3e9474cc0f39d2";

    @Autowired
    private MongoTemplate mongoTemplate;

    @Container
    @ServiceConnection
    private static final MongoDBContainer mongoDbContainer = new MongoDBContainer("mongo:6.0");

    @BeforeEach
    void setUp() {

        List<Entity> entities = new ArrayList<>();

        entities.addAll(List.of(
                new Entity(ENTITY_ID_1),
                new Entity(ENTITY_ID_2)

        ));

        mongoTemplate.insertAll(entities);
    }

    // Test if the find method works
    @Test
    void matchById_entity1() {

        final Criteria byEntityId = new Criteria("_id").is(ENTITY_ID_1);
        final MatchOperation matchStage = Aggregation.match(byEntityId);
        Aggregation aggregation = Aggregation.newAggregation(matchStage);

        List<Entity> entities = mongoTemplate.aggregate(aggregation, "testEntity", Entity.class).getMappedResults();

        assertThat(entities, is(notNullValue()));
        assertThat(entities, is(hasSize(1)));
        assertThat(entities.get(0).get_id(), is(ENTITY_ID_1));
    }

    @Test
    void matchById_entity2() {

        final Criteria byEntityId = new Criteria("_id").is(ENTITY_ID_2);
        final MatchOperation matchStage = Aggregation.match(byEntityId);
        Aggregation aggregation = Aggregation.newAggregation(matchStage);

        List<Entity> entities = mongoTemplate.aggregate(aggregation, "testEntity", Entity.class).getMappedResults();

        assertThat(entities, is(notNullValue()));
        assertThat(entities, is(hasSize(1)));
        assertThat(entities.get(0).get_id(), is(ENTITY_ID_2));
    }

    @AfterEach
    void tearDown() {
        mongoTemplate.dropCollection(Entity.class);
    }

    @Document(collection = "testEntity")
    public class Entity {

        @MongoId
        private String _id;

        public Entity(String _id) {
            this._id = _id;
        }

        public String get_id() {
            return _id;
        }
    }
}

Using typed aggregations also fixed this case:

Aggregation aggregation = Aggregation.newAggregation(Entity.class, matchStage);

While this resolved the issue for me, I thought it was still worth mentioning here as it may highlight a deeper problem or help creating better test coverage.