spring-projects / spring-data-couchbase

Provides support to increase developer productivity in Java when using Couchbase. Uses familiar Spring concepts such as a template classes for core API usage and lightweight repository style data access.
https://spring.io/projects/spring-data-couchbase
Apache License 2.0
277 stars 191 forks source link

Following DTO Projection doc #1543

Open rbleuse opened 2 years ago

rbleuse commented 2 years ago

Hello,

I was following the [DTO Projection](https://docs.spring.io/spring-data/couchbase/docs/current/reference/html/#couchbase.ansijoins:~:text=Iterable%3CAirport%3E%20findAll()%3B%0A%0A%7D-,5.3.4.%20DTO%20Projections,-Spring%20Data%20Repositories) documentation but can't make it work.

My set up : spring-boot 2.7.3 (data couchbase 4.4.2)

First, this documentation has example using spring data JPA (@Entity and @OneToOne annotations). Is it normal ?

Second, here is my simple test and the exception I get :

Person document :

@Document
@Scope("dev")
@Collection("person")
data class Person(
    @field:Id
    val id: String,

    @field:Field
    val name: String
)

Projection interface :

interface Name {
    fun getName(): String
}

Person repository :

interface PersonRepository : ReactiveCouchbaseRepository<Person, String> {
    fun findByName(name: String): Flux<Name>
}

Exception :

java.lang.IllegalArgumentException: returnType must not be null!
    at org.springframework.util.Assert.notNull(Assert.java:201) ~[spring-core-5.3.22.jar:5.3.22]
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ HTTP POST "/person" [ExceptionHandlingWebHandler]
Original Stack Trace:
        at org.springframework.util.Assert.notNull(Assert.java:201) ~[spring-core-5.3.22.jar:5.3.22]
        at org.springframework.data.couchbase.core.ReactiveFindByQueryOperationSupport$ReactiveFindByQuerySupport.as(ReactiveFindByQueryOperationSupport.java:134) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
        at org.springframework.data.couchbase.repository.query.AbstractReactiveCouchbaseQuery.lambda$getExecutionToWrap$1(AbstractReactiveCouchbaseQuery.java:122) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
        at org.springframework.data.couchbase.repository.query.ReactiveCouchbaseQueryExecution$ResultProcessingExecution.execute(ReactiveCouchbaseQueryExecution.java:77) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
        at org.springframework.data.couchbase.repository.query.AbstractReactiveCouchbaseQuery.doExecute(AbstractReactiveCouchbaseQuery.java:91) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
        at org.springframework.data.couchbase.repository.query.AbstractCouchbaseQueryBase.execute(AbstractCouchbaseQueryBase.java:133) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
        at org.springframework.data.couchbase.repository.query.AbstractCouchbaseQueryBase.execute(AbstractCouchbaseQueryBase.java:113) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:137) ~[spring-data-commons-2.7.2.jar:2.7.2]
        at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:121) ~[spring-data-commons-2.7.2.jar:2.7.2]
        at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:160) ~[spring-data-commons-2.7.2.jar:2.7.2]
        at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:139) ~[spring-data-commons-2.7.2.jar:2.7.2]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar:5.3.22]
        at org.springframework.data.couchbase.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:141) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar:5.3.22]
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-5.3.22.jar:5.3.22]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar:5.3.22]
        at org.springframework.data.repository.core.support.MethodInvocationValidator.invoke(MethodInvocationValidator.java:99) ~[spring-data-commons-2.7.2.jar:2.7.2]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.22.jar:5.3.22]
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.22.jar:5.3.22]
        at jdk.proxy2/jdk.proxy2.$Proxy101.findByName(Unknown Source) ~[na:na]
        at com.rbleuse.spring.reactive.couchbase.service.PersonService.getProjectionByName(PersonService.kt:13) ~[main/:na]

Is the documentation is up to date, and can we use projection with spring data couchbase ?

mikereiche commented 2 years ago

Looks like as() is being passed a null.

133                 public <R> FindByQueryWithConsistency<R> as(Class<R> returnType) {
134                         Assert.notNull(returnType, "returnType must not be null!");

It would be helpful to see com.rbleuse.spring.reactive.couchbase.service.PersonService.getProjectionByName(PersonService.kt:13)

mikereiche commented 2 years ago

Also :

If the domain class is annotated with the module-specific type annotation, it is a valid candidate for the particular Spring Data module. Spring Data modules accept either third-party annotations (such as JPA’s @Entity) or provide their own annotations

I'm not sure about OnetoOne.

rbleuse commented 2 years ago

It would be helpful to see com.rbleuse.spring.reactive.couchbase.service.PersonService.getProjectionByName(PersonService.kt:13)

Just a simple service :

@Service
class PersonService(
    private val repository: PersonRepository
) {

    fun createPerson(person: Person) = repository.save(person)
    fun getProjectionByName(name: String) = repository.findByName(name)
}

I updated my dummy repository on https://github.com/rbleuse/spring-reactive-couchbase/tree/projection (branch projection) if you would like to investigate on it

mikereiche commented 2 years ago

edit: getTypeToRead() is returning null even though it has enough information to determine the return type. I'm still investigating.

124:    private Object execute(ParametersParameterAccessor parameterAccessor) {

        Class<?> typeToRead = processor.getReturnedType().getTypeToRead();

As a work-around when only one simple property is being returned (in your case Name contains only a String), the method can be defined to return that simple time and it will work

fun findByName(name: String): Flux<String>

mikereiche commented 2 years ago

Name needs to be a class. This will suffice:

class Name(var name: String) {
}

It seems that some better diagnostics would help.

rbleuse commented 2 years ago

Thanks, indeed with a class it's working.

However I tried to proceed to the same with a custom n1ql instead of using the method name, but I faced this exception :

interface PersonRepository : ReactiveCouchbaseRepository<Person, String> {
    @Query("select p.firstName, p.lastName from #{#n1ql.bucket} p WHERE p.firstName = '#{[0]}' AND p.#{#n1ql.filter}")
    fun findByFirstName(firstName: String): Flux<PersonName>
}
@Document
@Scope("dev")
@Collection("person")
data class Person(
    @field:Id
    val id: String,

    @field:Field
    val firstName: String,

    @field:Field
    val lastName: String
)
data class PersonName(val firstName: String, val lastName: String)
stack trace ``` com.couchbase.client.core.error.CouchbaseException: __id was null. Either use #{#n1ql.selectEntity} or project __id at org.springframework.data.couchbase.core.ReactiveCouchbaseTemplateSupport.lambda$decodeEntity$5(ReactiveCouchbaseTemplateSupport.java:117) ~[spring-data-couchbase-4.4.2.jar:4.4.2] Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: Error has been observed at the following site(s): *__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ HTTP POST "/person" [ExceptionHandlingWebHandler] Original Stack Trace: at org.springframework.data.couchbase.core.ReactiveCouchbaseTemplateSupport.lambda$decodeEntity$5(ReactiveCouchbaseTemplateSupport.java:117) ~[spring-data-couchbase-4.4.2.jar:4.4.2] at reactor.core.publisher.MonoSupplier.call(MonoSupplier.java:86) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:405) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onNext(MonoFlatMapMany.java:250) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxRefCount$RefCountInner.onNext(FluxRefCount.java:200) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxPublish$PublishSubscriber.drain(FluxPublish.java:477) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxPublish$PublishSubscriber.onNext(FluxPublish.java:268) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.UnicastProcessor.drainRegular(UnicastProcessor.java:388) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.UnicastProcessor.drain(UnicastProcessor.java:470) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.UnicastProcessor.subscribe(UnicastProcessor.java:534) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxPublish.connect(FluxPublish.java:100) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxRefCount.subscribe(FluxRefCount.java:85) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.Flux.subscribe(Flux.java:8466) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:195) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:113) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:113) ~[reactor-core-3.4.22.jar:3.4.22] at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) ~[reactor-core-3.4.22.jar:3.4.22] at com.couchbase.client.core.Reactor$SilentMonoCompletionStage.lambda$subscribe$0(Reactor.java:183) ~[core-io-2.3.3.jar:na] at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[na:na] at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[na:na] at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[na:na] at com.couchbase.client.core.msg.BaseRequest.succeed(BaseRequest.java:161) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.io.netty.chunk.ChunkedMessageHandler.completeInitialResponse(ChunkedMessageHandler.java:274) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.io.netty.chunk.ChunkedMessageHandler.handleHttpContent(ChunkedMessageHandler.java:261) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.io.netty.chunk.ChunkedMessageHandler.channelRead(ChunkedMessageHandler.java:210) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[core-io-2.3.3.jar:na] at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[core-io-2.3.3.jar:na] at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na] ```

So it seems with a custom query we have to query the document id as well ? I have a concrete use case which is not that one, but this is a minimal repro of the issue I'm facing with my use case

mikereiche commented 2 years ago

Yes. An id is(was) always expected for an entity. Also cas. It doesn't matter what it is though. You can have "" as id, 0 as cas.

There is a change to only require id and cas if they are needed : https://github.com/spring-projects/spring-data-couchbase/issues/1402

That shows commits in main (May 3), 4.4.x (May 9) and 4.3.x (May 11). So it will be in the releases of those which occurred after (June 20 and July 15) https://calendar.spring.io/. July 15 shows 2021.1.6 -> 4.3.6, 2021.2.2 -> 4.4.2 and 2022.0.0 -> 5.0.0-M5.

rbleuse commented 2 years ago

Oh understood, so under the hood fun findByName(name: String): Flux<Name> will fetch the id even though it's not mapped in my projection dto

And for a manual custom query projection, I also need to add an __id to my query even if I don't need it.

Indeed it's working as expected if I don't declare it in my projection dto as long as I add the id in my query. Thank you !

mikereiche commented 2 years ago

I also need to add an __id to my query even if I don't need it.

Or you could just use the newer version that doesn't require it.

rbleuse commented 2 years ago

Which newer version are you referring to ?

I'm using 4.4.2 and it's still required with latest 4.4.3-SNAPSHOT

select p.firstName, p.lastName from #{#n1ql.bucket} p WHERE p.firstName = $1 AND p.#{#n1ql.filter}

com.couchbase.client.core.error.CouchbaseException: __id was null. Either use #{#n1ql.selectEntity} or project __id
    at org.springframework.data.couchbase.core.ReactiveCouchbaseTemplateSupport.lambda$decodeEntity$5(ReactiveCouchbaseTemplateSupport.java:117) ~[spring-data-couchbase-4.4.3-SNAPSHOT.jar:4.4.3-SNAPSHOT]
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ HTTP POST "/person" [ExceptionHandlingWebHandler]
mikereiche commented 2 years ago

My mistake - the id is always required when anything other than one simple field is projected. So just project "" as __id.