search

GoogleCloudPlatform / spring-cloud-gcp

New home for Spring Cloud GCP development starting with version 2.0.
Apache License 2.0
414 stars 305 forks source link

SpannerTemplate#update ambiguously requires primary keys in includeProperties #1987

Open austinmilt opened 1 year ago

austinmilt commented 1 year ago

Is your feature request related to a problem? Please describe. I was getting a NOT_FOUND error from the spanner emulator because I was not including the primary keys of my POJO in includeProperties in SpannerTemplate#update(Object object, Set<String> includeProperties). Documentation (primarily this) does not say this is necessary. Since primary keys are immutable, it implies that primary keys should not be in includeProperties.

Setup Spring Cloud Spanner

Cloud Spanner Emulator

Describe the solution you'd like Require that primary keys be defined - not null - on the update POJO but not in includeProperties. I think the easiest way to accomplish this is to add the primary keys to includeProperties within SpannerTemplate#update or SpannerMutationFactoryImpl#update

Describe alternatives you've considered Clearly document in Spring Cloud GCP that primary keys must be in includeProperties, ideally in both the inline docs and API docs.

Additional context The specific error is io.grpc.StatusRuntimeException: NOT_FOUND: Table users: Row {String(NULL)} not found.

Example of Mutation#toString created by Spring Cloud Spanner with and without putting the primary keys in includeProperties:

lqiu96 commented 1 year ago

Hi @austinmilt, thanks for bringing this up. Do you have a code sample that I can try to replicate this issue with? I want to confirm that including the primary key is required or if there is something that might be requiring the primary key to be included.

austinmilt commented 1 year ago

Hi @austinmilt, thanks for bringing this up. Do you have a code sample that I can try to replicate this issue with? I want to confirm that including the primary key is required or if there is something that might be requiring the primary key to be included.

Sure! See if this works for you: https://github.com/austinmilt/gcp-spanner-issue-1987

Here's my stdout:

austin@NotARealPC:~/personal/gcp-spanner-issue-demo$ mvn spring-boot:run
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for com.example:demo:jar:0.0.1-SNAPSHOT
[WARNING] 'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: com.google.cloud:spring-cloud-gcp-dependencies:pom -> version ${spring-cloud.version} vs ${spring-cloud-gcp.version} @ line 73, column 16
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING] 
[INFO] 
[INFO] --------------------------< com.example:demo >--------------------------
[INFO] Building demo 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] >>> spring-boot-maven-plugin:2.7.13:run (default-cli) > test-compile @ demo >>>
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ demo ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ demo ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/austin/personal/gcp-spanner-issue-demo/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ demo ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] skip non existing resourceDirectory /home/austin/personal/gcp-spanner-issue-demo/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ demo ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/austin/personal/gcp-spanner-issue-demo/target/test-classes
[INFO] 
[INFO] <<< spring-boot-maven-plugin:2.7.13:run (default-cli) < test-compile @ demo <<<
[INFO] 
[INFO] 
[INFO] --- spring-boot-maven-plugin:2.7.13:run (default-cli) @ demo ---
[INFO] Attaching agents: []

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::               (v2.7.13)

2023-06-29 15:50:59.383  INFO 115639 --- [           main] com.example.demo.DemoApplication         : Starting DemoApplication using Java 17.0.7 on NotARealPC with PID 115639 (/home/austin/personal/gcp-spanner-issue-demo/target/classes started by austin in /home/austin/personal/gcp-spanner-issue-demo)
2023-06-29 15:50:59.384  INFO 115639 --- [           main] com.example.demo.DemoApplication         : No active profile set, falling back to 1 default profile: "default"
2023-06-29 15:50:59.599  INFO 115639 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Spanner repositories in DEFAULT mode.
2023-06-29 15:50:59.604  INFO 115639 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 2 ms. Found 0 Spanner repository interfaces.
2023-06-29 15:50:59.857  INFO 115639 --- [           main] c.g.c.s.a.c.GcpContextAutoConfiguration  : The default project ID is demo
EXAMPLE STARTED
CREATING USERS TABLE
ADDING INITIAL VALUE
TRYING UPDATE WITHOUT PRIMARY KEYS
com.google.cloud.spanner.SpannerException: NOT_FOUND: io.grpc.StatusRuntimeException: NOT_FOUND: Table users: Row {String(NULL)} not found.
        at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:291)
        at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:297)
        at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:170)
        at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:110)
        at com.google.cloud.spanner.TransactionRunnerImpl$TransactionContextImpl.commit(TransactionRunnerImpl.java:298)
        at com.google.cloud.spanner.TransactionRunnerImpl.lambda$runInternal$0(TransactionRunnerImpl.java:1037)
        at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
        at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
        at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
        at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted(SpannerRetryHelper.java:79)
        at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted(SpannerRetryHelper.java:68)
        at com.google.cloud.spanner.TransactionRunnerImpl.runInternal(TransactionRunnerImpl.java:1058)
        at com.google.cloud.spanner.TransactionRunnerImpl.run(TransactionRunnerImpl.java:963)
        at com.google.cloud.spanner.SessionImpl.writeWithOptions(SessionImpl.java:145)
        at com.google.cloud.spanner.SessionPool$PooledSession.writeWithOptions(SessionPool.java:1398)
        at com.google.cloud.spanner.SessionPool$PooledSessionFuture.writeWithOptions(SessionPool.java:1152)
        at com.google.cloud.spanner.DatabaseClientImpl.lambda$writeWithOptions$0(DatabaseClientImpl.java:79)
        at com.google.cloud.spanner.DatabaseClientImpl.runWithSessionRetry(DatabaseClientImpl.java:236)
        at com.google.cloud.spanner.DatabaseClientImpl.writeWithOptions(DatabaseClientImpl.java:79)
        at com.google.cloud.spanner.DatabaseClientImpl.write(DatabaseClientImpl.java:70)
        at com.google.cloud.spring.data.spanner.core.SpannerTemplate.lambda$applyMutations$25(SpannerTemplate.java:629)
        at com.google.cloud.spring.data.spanner.core.SpannerTemplate.doWithOrWithoutTransactionContext(SpannerTemplate.java:724)
        at com.google.cloud.spring.data.spanner.core.SpannerTemplate.applyMutations(SpannerTemplate.java:623)
        at com.google.cloud.spring.data.spanner.core.SpannerTemplate.applySaveMutations(SpannerTemplate.java:375)
        at com.google.cloud.spring.data.spanner.core.SpannerTemplate.update(SpannerTemplate.java:333)
        at com.example.demo.DemoApplication.example(DemoApplication.java:55)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:920)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:731)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:307)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292)
        at com.example.demo.DemoApplication.main(DemoApplication.java:30)
Caused by: com.google.cloud.spanner.SpannerException: NOT_FOUND: io.grpc.StatusRuntimeException: NOT_FOUND: Table users: Row {String(NULL)} not found.
        at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:291)
        at com.google.cloud.spanner.SpannerExceptionFactory.fromApiException(SpannerExceptionFactory.java:311)
        at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:174)
        at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:110)
        at com.google.cloud.spanner.TransactionRunnerImpl$TransactionContextImpl$CommitRunnable.lambda$run$0(TransactionRunnerImpl.java:408)
        at io.opencensus.trace.CurrentSpanUtils$RunnableInSpan.run(CurrentSpanUtils.java:125)
        at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
        at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
        at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
        at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
        at com.google.api.gax.retrying.BasicRetryingFuture.handleAttempt(BasicRetryingFuture.java:200)
        at com.google.api.gax.retrying.CallbackChainRetryingFuture$AttemptCompletionListener.handle(CallbackChainRetryingFuture.java:135)
        at com.google.api.gax.retrying.CallbackChainRetryingFuture$AttemptCompletionListener.run(CallbackChainRetryingFuture.java:117)
        at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
        at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
        at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
        at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
        at com.google.api.core.AbstractApiFuture$InternalSettableFuture.setException(AbstractApiFuture.java:92)
        at com.google.api.core.AbstractApiFuture.setException(AbstractApiFuture.java:74)
        at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
        at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:84)
        at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
        at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
        at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
        at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
        at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
        at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:574)
        at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:544)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:541)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose(SpannerErrorInterceptor.java:100)
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567)
        at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.grpc.StatusRuntimeException: NOT_FOUND: Table users: Row {String(NULL)} not found.
        at io.grpc.Status.asRuntimeException(Status.java:539)
        ... 21 more
TRYING UPDATE WITH PRIMARY KEYS
EXAMPLE DONE
2023-06-29 15:51:00.590  INFO 115639 --- [           main] com.example.demo.DemoApplication         : Started DemoApplication in 1.413 seconds (JVM running for 1.556)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.817 s
[INFO] Finished at: 2023-06-29T15:51:00-07:00
[INFO] ------------------------------------------------------------------------
meltsufin commented 1 year ago

Is this happening with the real Spanner instance as well, or just the emulator?

austinmilt commented 1 year ago

Is this happening with the real Spanner instance as well, or just the emulator?

I havent spun up a real spanner instance yet. I'm trying to do most of my development locally before adding that expense.

burkedavison commented 4 months ago

@olavloite : Do you know if this is/was a known Spanner emulator issue?