String ID not working for embeded objects [DATAMONGO-1988]

spring-projects-issues commented 6 years ago

Riasat Al Jamil opened DATAMONGO-1988 and commented

Issue: Repository functions not finding anything. Always empty.

Probable Cause: Using String ID for embeded documents.

Example:

BaseModel:

public class BaseModel
{
    @Id
    @Getter
    @Setter
    public String id;
}

Station:

@Data
@EqualsAndHashCode (callSuper = false)
@Document (collection = "stations")
public class Station extends BaseModel
{
    @Indexed
    private final int     extId;
    @Indexed
    private final String  name;
    @Indexed
    private final boolean validFrom;
    @Indexed
    private       boolean validTo = false;
}

Route:

@Data
@EqualsAndHashCode (callSuper = false)
@Document (collection = "routes")
public class Route extends BaseModel
{
    @Indexed
    private final Station from;
    @Indexed
    private final Station to;
}

RouteRepository:

public interface RouteRepository extends MongoRepository<Route, String>
{
    Optional<Route> findByFrom_IdAndTo_Id (ObjectId fromId, ObjectId toId);
    Optional<Route> findByTo_IdAndFrom_Id (String toId, String fromId);
    Optional<Route> findByFrom_ExtIdAndTo_ExtId (int fromExtId, int toExtId);
}

Parts of RoutePersistenceService:

public Optional<Route> findByFrom_IdAndTo_Id (ObjectId fromId, ObjectId toId)
{
    return routeRepository.findByFrom_IdAndTo_Id(fromId, toId);
}

public Optional<Route> findByTo_IdAndFrom_Id (String toId, String fromId)
{
    return routeRepository.findByTo_IdAndFrom_Id(toId, fromId);
}

public Optional<Route> findByFromExtIdToExtId (int fromExtId, int toExtId)
{
    return routeRepository.findByFrom_ExtIdAndTo_ExtId(fromExtId, toExtId);
}

public Optional<Route> findById (String id)
{
    return routeRepository.findById(id);
}

Now calling all four functions with valid data, that should find and return a valid Route object from database, 1. findByFrom_IdAndTo_Id works.

2. findByTo_IdAndFrom_Id does NOT work.

3. findByFromExtIdToExtId works as expected.

4. findById works as expected.

So there is a problem here. Namely, Using String as ID is not working well for embeded objects.

I believe there is a similar issue with @DBRef if you use int as ID instead of String. Updating such a document destroys the reference.

Affects: 2.1 M3 (Lovelace)

Issue Links:

DATAMONGO-1998 SpringDataMongodbSerializer cannot convert query on nested id
DATAMONGO-1798 Introduce @MongoId to customize Id conversion

Referenced from: pull request https://github.com/spring-projects/spring-data-mongodb/pull/565

Backported to: 2.0.8 (Kay SR8), 1.10.13 (Ingalls SR13)

spring-projects-issues commented 6 years ago

Riasat Al Jamil commented

It might be related to https://jira.spring.io/browse/DATAMONGO-1273

spring-projects-issues commented 6 years ago

Christoph Strobl commented

Thanks Riasat Al Jamil for reporting. I was not able to reproduce the issue you describe. Maybe I'm missing something. The following worked just as expected.

class Route {
    @Id String id;
    Station from, to;
}

class Station {
    @Id String id;
    String name;
}

interface RouteRepository extends MongoRepository<Route, String> {
    Route findByFrom_IdAndTo_Id(String from, String to);
}

by executing from._id=s1, to._id=s2 against the route collection holding objects that look like.

{
    "_id" : "r1",
    "from" : {
        "_id" : "s1",
        "name" : "farcry"
    },
    "to" : {
        "_id" : "s2",
        "name" : "city"
    },
    "_class" : "example.springframework.data.mongodb.Route"
}

Can you please add a samples of the documents within the store as well as the query executed. You can log the query via <logger name="org.springframework.data.mongodb.core.MongoTemplate" level="debug" />

spring-projects-issues commented 6 years ago

Riasat Al Jamil commented

Let's say I have this set of code:

Optional<Route> findByFrom_IdAndTo_Id (ObjectId fromId, ObjectId toId);
&mdash;
public Optional<Route> findByFrom_IdAndTo_Id (ObjectId fromId, ObjectId toId)
{
    return routeRepository.findByFrom_IdAndTo_Id(fromId, toId);
}
&mdash;
Optional<Station> fromOptional = stationPersistenceService.findById(fromStationId);
...
Optional<Station> toOptional = stationPersistenceService.findById(toStationId);
...
routeOptional = routePersistenceService.findByFrom_IdAndTo_Id(new ObjectId(fromStationId), new ObjectId(toStationId));

Trace log from

logging.level.org.springframework.data=TRACE

results in

2018-05-29 20:28:55.221 DEBUG 18260 --- [   XNIO-2 I/O-3] o.s.d.m.c.MongoTemplate                  : findOne using query: { "id" : "5b046430b2342942944a54db" } fields: {} for class: class [...]booking.persistence.models.Station in collection: stations
2018-05-29 20:28:55.224 DEBUG 18260 --- [   XNIO-2 I/O-3] o.s.d.m.c.MongoTemplate                  : findOne using query: { "_id" : { "$oid" : "5b046430b2342942944a54db" } } fields: { } in db.collection: booking.stations
2018-05-29 20:28:55.246 DEBUG 18260 --- [   XNIO-2 I/O-3] o.s.d.m.c.MongoTemplate                  : findOne using query: { "id" : "5b046430b2342942944a54d6" } fields: {} for class: class [...]booking.persistence.models.Station in collection: stations
2018-05-29 20:28:55.246 DEBUG 18260 --- [   XNIO-2 I/O-3] o.s.d.m.c.MongoTemplate                  : findOne using query: { "_id" : { "$oid" : "5b046430b2342942944a54d6" } } fields: { } in db.collection: booking.stations
2018-05-29 20:28:55.253 DEBUG 18260 --- [   XNIO-2 I/O-3] o.s.d.m.r.q.MongoQueryCreator            : Created query Query: { "from.id" : { "$oid" : "5b046430b2342942944a54db" }, "to.id" : { "$oid" : "5b046430b2342942944a54d6" } }, Fields: { }, Sort: { }
2018-05-29 20:28:55.256 DEBUG 18260 --- [   XNIO-2 I/O-3] o.s.d.m.c.MongoTemplate                  : find using query: { "from._id" : { "$oid" : "5b046430b2342942944a54db" }, "to._id" : { "$oid" : "5b046430b2342942944a54d6" } } fields: {} for class: class [...]booking.persistence.models.Route in collection: routes

which is as expected I guess...

Now I change the code to this (from ObjectId to String):

Optional<Route> findByFrom_IdAndTo_Id (String fromId, String toId);
&mdash;
public Optional<Route> findByFrom_IdAndTo_Id (String fromId, String toId)
{
    return routeRepository.findByFrom_IdAndTo_Id(fromId, toId);
}
&mdash;
Optional<Station> fromOptional = stationPersistenceService.findById(fromStationId);
...
Optional<Station> toOptional = stationPersistenceService.findById(toStationId);
...
routeOptional = routePersistenceService.findByFrom_IdAndTo_Id(fromStationId, toStationId);

and the log output is this:

2018-05-29 20:33:04.295 DEBUG 9936 --- [   XNIO-2 I/O-8] o.s.d.m.c.MongoTemplate                  : findOne using query: { "id" : "5b046430b2342942944a54db" } fields: {} for class: class [...]booking.persistence.models.Station in collection: stations
2018-05-29 20:33:04.299 DEBUG 9936 --- [   XNIO-2 I/O-8] o.s.d.m.c.MongoTemplate                  : findOne using query: { "_id" : { "$oid" : "5b046430b2342942944a54db" } } fields: { } in db.collection: booking.stations
2018-05-29 20:33:04.327 DEBUG 9936 --- [   XNIO-2 I/O-8] o.s.d.m.c.MongoTemplate                  : findOne using query: { "id" : "5b046430b2342942944a54d6" } fields: {} for class: class [...]booking.persistence.models.Station in collection: stations
2018-05-29 20:33:04.327 DEBUG 9936 --- [   XNIO-2 I/O-8] o.s.d.m.c.MongoTemplate                  : findOne using query: { "_id" : { "$oid" : "5b046430b2342942944a54d6" } } fields: { } in db.collection: booking.stations
2018-05-29 20:33:04.335 DEBUG 9936 --- [   XNIO-2 I/O-8] o.s.d.m.r.q.MongoQueryCreator            : Created query Query: { "from.id" : "5b046430b2342942944a54db", "to.id" : "5b046430b2342942944a54d6" }, Fields: { }, Sort: { }
2018-05-29 20:33:04.339 DEBUG 9936 --- [   XNIO-2 I/O-8] o.s.d.m.c.MongoTemplate                  : find using query: { "from._id" : "5b046430b2342942944a54db", "to._id" : "5b046430b2342942944a54d6" } fields: {} for class: class [...]booking.persistence.models.Route in collection: routes

The String id is getting converted to $oid in findById method, but not in findByFrom_IdAndTo_Id method which involves embeded objects.

PS: Why does it always search twice though?

spring-projects-issues commented 6 years ago

Riasat Al Jamil commented

build.gradle file if needed:

buildscript {
    ext {
        springBootVersion = '2.0.2.RELEASE'
    }
    repositories {
        jcenter()
        mavenCentral()
        maven { url "https://repo.spring.io/snapshot" }
        maven { url "https://repo.spring.io/milestone" }
        maven { url 'https://repo.spring.io/plugins-release' }
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
        classpath 'io.spring.gradle:propdeps-plugin:0.0.9.RELEASE'
        classpath 'com.jfrog.bintray.gradle:gradle-bintray-plugin:1.7.3'
        classpath "org.jfrog.buildinfo:build-info-extractor-gradle:4+"
    }
}

apply plugin: 'idea'
apply plugin: 'java'
apply plugin: 'maven-publish'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'
apply plugin: 'propdeps'
apply plugin: 'propdeps-idea'
apply plugin: 'com.jfrog.bintray'
apply plugin: "com.jfrog.artifactory"

group = 'com.[...].services'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.9

repositories {
    jcenter()
    mavenCentral()
    maven { url "https://repo.spring.io/snapshot" }
    maven { url "https://repo.spring.io/milestone" }
    maven {
        credentials {
            username = "${artifactory_user}"
            password = "${artifactory_password}"
        }
        url "http://artifacts.[...]/artifactory/[...]-commons"
    }
}

configurations {
    //compile.exclude module: 'spring-boot-starter-reactor-netty'
    compile.exclude module: 'spring-boot-starter-logging'
    compile.exclude module: 'spring-boot-starter-web'
    compile.exclude group: 'ch.qos.logback'
}

task wrapper(type: Wrapper) {
    gradleVersion = '4.6.0'
}

bootJar {
    launchScript()
}

ext {
    springCloudVersion = 'Finchley.BUILD-SNAPSHOT'
}

dependencies {
    compile(
            // Spring (core) libraies
            'org.springframework.boot:spring-boot-starter-actuator:2.0.1.RELEASE',
            'org.springframework.boot:spring-boot-starter-cache',
            'org.springframework.boot:spring-boot-starter-webflux',
            'org.springframework.boot:spring-boot-starter-log4j2',
            'org.springframework.boot:spring-boot-starter-undertow',
            'org.springframework.boot:spring-boot-starter-security',

            // Non-core spring libraries, their versions do NOT scale with the boot release
            'org.springframework.cloud:spring-cloud-starter-config:2.0.0.M9',
            'org.springframework.cloud:spring-cloud-starter-consul-discovery:2.0.0.M7',
            'org.springframework.cloud:spring-cloud-starter-consul-config:2.0.0.M7',
            'org.springframework.data:spring-data-mongodb:2.1.0.M1',
            'org.springframework.cloud:spring-cloud-starter-aws:2.0.0.M4',
            'org.springframework.cloud:spring-cloud-starter-aws-messaging:2.0.0.M4',

            // Generic libraries
            // JWT
            'io.jsonwebtoken:jjwt:0.9.0',
            // Disruptor (for Log4j2 async)
            'com.lmax:disruptor:3.3.7',
            // Caching
            'com.hazelcast:hazelcast-spring:3.9.2',
            'com.github.ben-manes.caffeine:caffeine',
            // Serialization (JSON)
            'com.fasterxml.jackson.module:jackson-module-afterburner:2.9.3',
            // XML Serialization
            "javax.xml.bind:jaxb-api:2.3.0",
            // DAO
            // Password Strength Checking
            'com.nulab-inc:zxcvbn:1.2.3',
            // Search
            //'org.elasticsearch:elasticsearch:6.2.2',
            //'org.elasticsearch.client:transport:6.2.2',
            //'org.elasticsearch.plugin:transport-netty4-client:6.2.2',

            // Language Utilities
            'org.apache.commons:commons-lang3:3.7',

            // Commons library (MS Base)
            'com.[...].services:commons:0.0.3-SNAPSHOT'
    )
    optional "org.springframework.boot:spring-boot-configuration-processor"
    compileOnly('org.projectlombok:lombok:1.16.20')
    testCompile(
            'org.springframework.boot:spring-boot-starter-test',
            'io.projectreactor:reactor-test',
            'org.springframework.security:spring-security-test'
    )
}

dependencyManagement {
    imports {
        mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
    }
}

artifactory {
    contextUrl = "${artifactory_contextUrl}"   //The base Artifactory URL if not overridden by the publisher/resolver
    publish {
        repository {
            repoKey = '[...]-services'
            username = "${artifactory_user}"
            password = "${artifactory_password}"
            maven = true
        }

        defaults {
            publications('mavenJava')
        }
    }
}

publishing {
    publications {
        mavenJava(MavenPublication) {
            from components.java
        }
    }
}

artifactoryPublish {
    dependsOn bootJar
}

spring-projects-issues commented 6 years ago

Christoph Strobl commented

thanks Riasat Al Jamil. Seem the culprit is in the String to ObjectId conversion that converts any String used for an id property that represents a valid ObjectId into the such. However the QueryMapper seems to not recognize the embedded field as a top level id and therfore skips this conversation. We'll have a closer look

spring-projects / spring-data-mongodb

String ID not working for embeded objects [DATAMONGO-1988] #2859