Unable to read rows with null fields: "NullPointerException: Column ... contains NULL value"

[adeshmukh-quizlet]

I have a field mapped to nullable column in my entity class (using Kotlin).

        @Column(name = "field_a", spannerType = INT64, nullable = true)
        var fieldA: Short?,

(I am aware that the nullable = true only applies to schema generation after reading https://github.com/spring-cloud/spring-cloud-gcp/issues/1078)

I am able to save records with null values for this field. However, when I attempt to query the spanner table, I encounter an NPE:

Caused by: java.lang.NullPointerException: Column field_a contains NULL value
    at com.google.cloud.spanner.AbstractStructReader.checkNonNull(AbstractStructReader.java:336) ~[google-cloud-spanner-1.4.0.jar:1.4.0]
    at com.google.cloud.spanner.AbstractStructReader.checkNonNullOfType(AbstractStructReader.java:348) ~[google-cloud-spanner-1.4.0.jar:1.4.0]
    at com.google.cloud.spanner.AbstractStructReader.getLong(AbstractStructReader.java:113) ~[google-cloud-spanner-1.4.0.jar:1.4.0]
    at org.springframework.cloud.gcp.data.spanner.core.convert.StructAccessor.getSingleValue(StructAccessor.java:136) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]
    at org.springframework.cloud.gcp.data.spanner.core.convert.StructPropertyValueProvider.readSingleWithConversion(StructPropertyValueProvider.java:104) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]
    at org.springframework.cloud.gcp.data.spanner.core.convert.StructPropertyValueProvider.getPropertyValue(StructPropertyValueProvider.java:88) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]
    at org.springframework.cloud.gcp.data.spanner.core.convert.StructPropertyValueProvider.getPropertyValue(StructPropertyValueProvider.java:38) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]
    at org.springframework.data.mapping.model.PersistentEntityParameterValueProvider.getParameterValue(PersistentEntityParameterValueProvider.java:71) ~[spring-data-commons-2.1.2.RELEASE.jar:2.1.2.RELEASE]
    at org.springframework.data.convert.ClassGeneratingEntityInstantiator$EntityInstantiatorAdapter.extractInvocationArguments(ClassGeneratingEntityInstantiator.java:246) ~[spring-data-commons-2.1.2.RELEASE.jar:2.1.2.RELEASE]
    at org.springframework.data.convert.ClassGeneratingEntityInstantiator$EntityInstantiatorAdapter.createInstance(ClassGeneratingEntityInstantiator.java:219) ~[spring-data-commons-2.1.2.RELEASE.jar:2.1.2.RELEASE]
    at org.springframework.data.convert.ClassGeneratingEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:84) ~[spring-data-commons-2.1.2.RELEASE.jar:2.1.2.RELEASE]
    at org.springframework.cloud.gcp.data.spanner.core.convert.ConverterAwareMappingSpannerEntityReader.read(ConverterAwareMappingSpannerEntityReader.java:92) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]
    at org.springframework.cloud.gcp.data.spanner.core.convert.ConverterAwareMappingSpannerEntityProcessor.mapToList(ConverterAwareMappingSpannerEntityProcessor.java:79) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]
    at org.springframework.cloud.gcp.data.spanner.core.SpannerTemplate.mapToListAndResolveChildren(SpannerTemplate.java:480) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]
    at org.springframework.cloud.gcp.data.spanner.core.SpannerTemplate.query(SpannerTemplate.java:176) ~[spring-cloud-gcp-data-spanner-1.1.0.RELEASE.jar:1.1.0.RELEASE]

How do I go about reading null values in this case?

[adeshmukh-quizlet]

I should also point out that my entity class has 2 constructors, one that takes all the properties and another that only takes the "required" properties.

[meltsufin]

@adeshmukh-quizlet Thanks for the issue submission. We'll take a look at it shortly. @ChengyuanZhao PTAL.

[ChengyuanZhao]

@adeshmukh-quizlet thanks much for this report! investigating right now.

[ChengyuanZhao]

@adeshmukh-quizlet I'm unable to reproduce the problem in Java (actually we cover this NULL write-then-read usage in SpannerRepositoryIntegrationTests lines 195 and 196).

I'm unfamiliar with Kotlin; would you be able to set some breakpoints and help me debug?
In the class ConverterAwareMappingSpannerEntityReader there is a method shouldSkipProperty that checks whether that specific field is NULL in the struct and would then skip reading it in that case. Specifically it checks with the line struct.isNull(columnName). If you were to set a breakpoint with your kotlin code, can you verify that the state of the struct, the column name, and the contents of that specific column in the struct are as expected?

EDIT: Actually perhaps it's the Spring Data constructors "magic" that doesn't use the shouldSkipProperty test. Could you give more details on the constructors you're using? .

[adeshmukh-quizlet]

@ChengyuanZhao Thanks for your prompt attention to this issue. Here's a screenshot of the debugging attempt. The breakpoint is at one of the calls in the NPE stacktrace. I note the following:

1. The instantiator used here is KotlinClassGeneratingEntityInstantiator.
2. The NPE happens before the shouldSkipProperty(...) check is attempted (at line 103 in the screenshot).

[adeshmukh-quizlet]

Further debugging led me to the spring-data-commons library which instantiates the entity class. The Kotlin implementation looks for a constructor with @PreferredConstructor to instantiate the entity instance (Ref: spring-data-commons:PreferredConstructorDiscoverer.java).

I was able to resolve this for my case by adding a default constructor to my Kotlin entity data class and annotating it with @PreferredConstructor.

[ChengyuanZhao]

@adeshmukh-quizlet ah very interesting! thank you for these clues.

I'm still trying to make the problem come up here, but can I ask you to try a small little fix to see if it resolves your situation ? (without using the PreferredConstructor annotation solution).

I think I know the cause: in StructPropertyValueProvider, there is a single public function getPropertyValue. the first thing this does is check for the existence of that column in the Spanner struct at all. Right after that if statement, could you see if adding this condition fixes it ?

So it would look like this:

...
if (!this.structAccessor.hasColumn(colName)) {
            throw new SpannerDataException("Column not found: " + colName);
        }
if (this.structAccessor.isNull(colName) {
            return null;
        }
...

[adeshmukh-quizlet]

Ok, so I think the root cause can be addressed in StructAccessor.java. The singleItemReadMethodMappingIntCol map registers the same function for primitive as well as wrapper types and that function always checks for non-null. I think it would be more appropriate to check for non-null only if the column type is primitive and allow nulls for the wrapper types.

[ChengyuanZhao]

@adeshmukh-quizlet That's a good observation , however the getter-functions on Struct only return the primitive types and always check for non-null (which is why I added the isNull method in StructAccessor). I think what was missing was a call to isNull in StructPropertyValueProvider as I describe in the previous post. What was being overlooked was Spring Data's constructor/instantiators that sidestep the shouldSkipProperty check and directly use StructPropertyValueProvider

[adeshmukh-quizlet]

Can the function return a non-primitive value instead? Because I observe that Object getSingleValue(String colName) returns a wrapper type anyway.

[ChengyuanZhao]

Unfortunately those column-getter functions are in the client lib, which is a different repo that is set in stone from the perspective of this one.

[adeshmukh-quizlet]

Ah, yeah - I see it now. That is delegated to google-cloud-spanner API. Your original suggestion makes sense then, to avoid the call to readSingleWithConversion(...) by checking for null value earlier.

... or implement a StructAccessor::getLongNullable and register that instead of AbstractStructReader::getLong for the wrapper types.

[ChengyuanZhao]

Making a PR now!