Closed martinrist closed 10 months ago
scrocquesel
commented
11 months ago Hello, I've reproduced the bug in dev and test mode. I suspect that the AWS SDK is not using the correct classloader. I need to delve deeper into the AWS SDK and Quarkus TCCL logic in dev/test mode.
Any assistance would be appreciated.
scrocquesel
commented
11 months ago enabling aws sdk debug log didn't produce much information but at least it shows that the TableSchema is cached successfully and it's only when trying to call the builder create method that something fails.
quarkus.log.category."software.amazon.awssdk.enhanced.dynamodb.beans".level=DEBUGI also added the classloader name for some classes and it differs
System.out.println(DynamoDBExampleTableEntryImmutable.class.getClassLoader().getName());
System.out.println(DynamoDBExampleTableEntryImmutable.class.getClassLoader().getParent().getName());
System.out.println(TableSchema
.fromClass(DynamoDBExampleTableEntryImmutable.class).getClass().getClassLoader().getName());
try {
System.out.println(TableSchema
.fromClass(DynamoDBExampleTableEntryImmutable.class).getClass().getClassLoader().loadClass(DynamoDBExampleTableEntryImmutable.class.getName()).getName());
} catch (ClassNotFoundException e) {
e.printStackTrace();
}Quarkus Runtime ClassLoader: DEV for quarkus-amazon-services-integration-tests-999-SNAPSHOT restart no:2
Quarkus Base Runtime ClassLoader: DEV for quarkus-amazon-services-integration-tests-999-SNAPSHOT
Quarkus Base Runtime ClassLoader: DEV for quarkus-amazon-services-integration-tests-999-SNAPSHOT
java.lang.ClassNotFoundException: io.quarkus.it.amazon.dynamodb.enhanced.DynamoDBExampleTableEntryImmutable
[...]Note that the Base CL is the parent of the first one and only the child CL can load the bean class.
martinrist
commented
11 months ago Posting this additional link in here that we discovered whilst initially trying to investigate this issue -> https://github.com/aws/aws-sdk-java-v2/issues/2604.
Although this is an old issue, and refers to the @DynamoDbBean version of the mapping (along with TableSchema.fromBean(...)), it might throw some light on what's happening.
Comments in that aws-sdk-java-v2 issue also link to this issue in the Quarkus repo. Again, that's an old (now fixed) issue with @DynamoDbBean mapping, but might be of use.
scrocquesel
commented
11 months ago @martinrist Thanks for the pointer. Some contributors did provide a fix for DynamoDbBean in the past to avoid lambda generated code and issue with dev/test mode CL.
https://github.com/quarkiverse/quarkus-amazon-services/blob/main/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java#L226
We need to replicate the behavior for DynamoDbImmutable. As a workaround, if you have a few small entities, you can declare the schema statically.
var schema = StaticImmutableTableSchema
.builder(DynamoDBExampleTableEntryImmutable.class, DynamoDBExampleTableEntryImmutable.Builder.class)
.newItemBuilder(DynamoDBExampleTableEntryImmutable::builder,
DynamoDBExampleTableEntryImmutable.Builder::build)
.addAttribute(String.class, a -> a.name("key_id")
.getter(DynamoDBExampleTableEntryImmutable::getKeyId)
.setter(DynamoDBExampleTableEntryImmutable.Builder::keyId)
.addTag(primaryPartitionKey()))
.addAttribute(String.class, a -> a.name("payload")
.getter(DynamoDBExampleTableEntryImmutable::getPayload)
.setter(DynamoDBExampleTableEntryImmutable.Builder::payload))
.addAttribute(String.class, a -> a.name("range_id")
.getter(DynamoDBExampleTableEntryImmutable::getRangeId)
.setter(DynamoDBExampleTableEntryImmutable.Builder::rangeId))
.addAttribute(Instant.class, a -> a.name("created_at_timestamp")
.getter(DynamoDBExampleTableEntryImmutable::getCreatedAtTimestamp)
.setter(DynamoDBExampleTableEntryImmutable.Builder::createdAtTimestamp))
.addAttribute(Instant.class, a -> a.name("updated_at")
.getter(DynamoDBExampleTableEntryImmutable::getUpdatedAt)
.setter(DynamoDBExampleTableEntryImmutable.Builder::updatedAt))
.addAttribute(Long.class, a -> a.name("version")
.getter(DynamoDBExampleTableEntryImmutable::getVersion)
.setter(DynamoDBExampleTableEntryImmutable.Builder::version))
.build();
DynamoDbTable<DynamoDBExampleTableEntryImmutable> exampleBlockingTableFromClient = dynamoEnhancedClient.table(
BLOCKING_TABLE,
schema);@scrocquesel - thanks for the update and the workaround suggestion. I've successfully applied that workaround to the example 'Quarkus Fruits' project posted on the original issue:
DynamoDbTable<Fruit> fruitTable = dynamoEnhancedClient.table(FRUIT_TABLE_NAME, buildSchema());
private TableSchema<Fruit> buildSchema() {
return StaticImmutableTableSchema
.builder(Fruit.class, Fruit.Builder.class)
.newItemBuilder(Fruit::builder, Fruit.Builder::build)
.addAttribute(String.class, a -> a.name("fruitName")
.getter(Fruit::name)
.setter(Fruit.Builder::name)
.addTag(primaryPartitionKey()))
.addAttribute(String.class, a -> a.name("fruitDescription")
.getter(Fruit::description)
.setter(Fruit.Builder::description))
.build();
}This workaround allows us to convert the Fruit model class to a plain Java record, together with Lombok's @Builder annotation, which removes duplication between model annotations and the static schema definition:
@Builder(builderClassName = "Builder")
public record Fruit(String name, String description) { }I made some progress but I'm stuck with the last call to build on the builder to materialize the entity and I guess I'll have to make PR in the AWS sdk repo to be able to hook in their code.
scrocquesel
commented
10 months ago @martinrist @cairngorm Can you test the main branch ? Unit tests passed but just want to be sure it works in real stituation before releasing a new version
cairngorm
commented
10 months ago @scrocquesel Yes. I should have time to look at this tomorrow.
cairngorm
commented
10 months ago @scrocquesel I tested the native build with your fix and it's working as expected. Previously, I had to add -Dquarkus.native.additional-build-args="--initialize-at-run-time=software.amazon.awssdk.enhanced.dynamodb.internal.mapper.LambdaToMethodBridgeBuilder" to the native build command. With your fix, that's no longer required.
Thank-you for your time investigating and fixing this issue :smile:
martinrist
commented
10 months ago @scrocquesel - @cairngorm & I have tested this update on both the sample project attached to this issue and on the original project we're both working on, and confirm that the main branch (as at commit 3c7758) addresses the issue.
Thanks for all your help on getting this resolved so quickly!
scrocquesel
commented
10 months ago I'll prepare the release after work this evening. Thank you for testing and raising this issue
scrocquesel
commented
10 months ago 2.7.3 is out
Overview
When using the DynamoDB Enhanced Client's
@DynamoDbBeanannotation to map a mutable Java class to a DynamoDB table, everything works as expected, both in Quarkus Dev mode and in production. Data is returned from DynamoDB and is mapped to the class correctly.However, when switching over to using an immutable class, with
@DynamoDbImmutableand a suitableBuilderimplementation, running in Quarkus Dev Mode produces the following stack trace when attempting to access an endpoint that accesses the same table via the immutable class:Doing the same in a 'production' profile works as expected for both the mutable and immutable class versions.
Example Repository and Reproduction Steps
Attached to the issue is a minimal sample that illustrates the issue. This is based on the 'Quarkus Fruits' example application described here - in particular the 'DynamoDB Enhanced Client' section of the guide.
In the example, classes in the
org.acme.dynamodb.mutablepackage contain aMutableFruitmodel object, mapped using@DynamoDbBean. When run in Quarkus Dev mode, this exposes an endpoint at http://localhost:8080/mutable-fruits, which behaves as expected.Classes in the
org.acme.dynamodb.immutablepackage have been updated to an immutable class, mapped with@DynamoDbImmutableas described in this AWS post. TheFruitmodel class has been made immutable using manually-written Java features. The same behaviour is observed if getters, and builders are generated using Lombok, or if Java records are used.When run in Quarkus Dev mode, this exposes an endpoint at http://localhost:8080/fruits. If the underlying DynamoDB table contains no records, this behaves correctly. However, if the table contains at least one record, the exception described above is observed.
Full instructions on setting up the example repository, including setting up a local DynamoDB instance with the expected table and records, can be found in the README file in the supplied example.
Build Details
Sample Project
quarkus-amazon-services-issue-1035-amazon-dynamodb-quickstart.zip