At long last, the wait for codegenned mapper operations and high-level types is ~done~ started! This change removes the handwritten toy implementation of a getItem operation and now generates the operation glue code for the following DDB ops:
deleteItem
getItem
putItem
query*
scan*
* These operations should be paginated but the codegen doesn't handle that yet. It's left as a FIXME in the code for follow-up.
PR flight plan
⚠️ There's a lot to take in here, especially if you're unfamiliar with KSP. I recommend taking a short break to learn some basics about KSP before getting started with the review.
Recommended order of review:
Start with the new ddb-mapper-ops-codegen module, in particular:
HighLevelOpsProcessor which is the main entry point for KSP. Everything else in this module is called somehow from this point.
The model package is for data types for describing the low- & high-level operations, structures, and types
The core package contains rudimentary code writing utilities such as indentation, blocks, imports, and template processing. These are good candidates for commonizing somewhere in the future, since we'll likely need similar functionality for DDB mapper annotation processing and possibly other HLLs in the future.
The rendering package contains structured codegen for rendering the operations, convenience functions, and types
Next, hit up the dynamodb-mapper module, in particular:
build.gradle.kts has been beefed up with the new KSP application of ddb-mapper-ops-codegen. Some unpleasant (and hopefully temporary) hacks were necessary to get KSP to work in both JVM+Native builds and JVM-only builds.
The new unit tests in the operations package which use DynamoDB Local in lieu of mocking or making actual calls to DDB
Sample codegen output
Understanding the codegen may be easier if you see the generated code. All code is generated into the hll/ddb-mapper/dynamodb-mapper/build/generated/ksp/common/commonMain/kotlin directory.
GetItem operation
```kotlin
// Code generated by ddb-mapper-ops-codegen. DO NOT EDIT!
package aws.sdk.kotlin.hll.dynamodbmapper.operations
import aws.sdk.kotlin.hll.dynamodbmapper.TableSpec
import aws.sdk.kotlin.hll.dynamodbmapper.items.ItemSchema
import aws.sdk.kotlin.hll.dynamodbmapper.model.toItem
import aws.sdk.kotlin.hll.dynamodbmapper.pipeline.internal.HReqContextImpl
import aws.sdk.kotlin.hll.dynamodbmapper.pipeline.internal.MapperContextImpl
import aws.sdk.kotlin.hll.dynamodbmapper.pipeline.internal.Operation
import aws.sdk.kotlin.services.dynamodb.model.ConsumedCapacity
import aws.sdk.kotlin.services.dynamodb.model.ReturnConsumedCapacity
import kotlin.Boolean
import kotlin.String
import aws.sdk.kotlin.services.dynamodb.model.GetItemRequest as LowLevelGetItemRequest
import aws.sdk.kotlin.services.dynamodb.model.GetItemResponse as LowLevelGetItemResponse
public interface GetItemRequest {
public companion object { }
public val consistentRead: Boolean?
public val key: T?
public val returnConsumedCapacity: ReturnConsumedCapacity?
}
private data class GetItemRequestImpl(
override val consistentRead: Boolean?,
override val key: T?,
override val returnConsumedCapacity: ReturnConsumedCapacity?,
): GetItemRequest
public fun GetItemRequest(
consistentRead: Boolean?,
key: T?,
returnConsumedCapacity: ReturnConsumedCapacity?,
): GetItemRequest = GetItemRequestImpl(
consistentRead,
key,
returnConsumedCapacity,
)
private fun GetItemRequest.convert(
tableName: String?,
schema: ItemSchema,
) = LowLevelGetItemRequest {
consistentRead = this@convert.consistentRead
returnConsumedCapacity = this@convert.returnConsumedCapacity
this@convert.key?.let { key = schema.converter.toItem(it, schema.keyAttributeNames) }
this.tableName = tableName
}
public interface GetItemResponse {
public companion object { }
public val consumedCapacity: ConsumedCapacity?
public val item: T?
}
private data class GetItemResponseImpl(
override val consumedCapacity: ConsumedCapacity?,
override val item: T?,
): GetItemResponse
public fun GetItemResponse(
consumedCapacity: ConsumedCapacity?,
item: T?,
): GetItemResponse = GetItemResponseImpl(
consumedCapacity,
item,
)
private fun LowLevelGetItemResponse.convert(schema: ItemSchema) = GetItemResponse(
consumedCapacity = this@convert.consumedCapacity,
item = this@convert.item?.toItem()?.let(schema.converter::fromItem),
)
internal fun getItemOperation(table: TableSpec) = Operation(
initialize = { hReq: GetItemRequest -> HReqContextImpl(hReq, table.schema, MapperContextImpl(table, "GetItem")) },
serialize = { hReq, schema -> hReq.convert(table.name, schema) },
lowLevelInvoke = table.mapper.client::getItem,
deserialize = LowLevelGetItemResponse::convert,
interceptors = table.mapper.config.interceptors,
)
```
Table operations interface/implementation
```kotlin
// Code generated by ddb-mapper-ops-codegen. DO NOT EDIT!
package aws.sdk.kotlin.hll.dynamodbmapper.operations
import aws.sdk.kotlin.hll.dynamodbmapper.TableSpec
/**
* Provides access to operations on a particular table, which will invoke low-level operations after
* mapping objects to items and vice versa
* @param T The type of objects which will be read from and/or written to this table
*/
public interface TableOperations {
public suspend fun deleteItem(request: DeleteItemRequest): DeleteItemResponse
public suspend fun getItem(request: GetItemRequest): GetItemResponse
public suspend fun putItem(request: PutItemRequest): PutItemResponse
public suspend fun query(request: QueryRequest): QueryResponse
public suspend fun scan(request: ScanRequest): ScanResponse
}
internal class TableOperationsImpl(private val tableSpec: TableSpec) : TableOperations {
override suspend fun deleteItem(request: DeleteItemRequest) =
deleteItemOperation(tableSpec).execute(request)
override suspend fun getItem(request: GetItemRequest) =
getItemOperation(tableSpec).execute(request)
override suspend fun putItem(request: PutItemRequest) =
putItemOperation(tableSpec).execute(request)
override suspend fun query(request: QueryRequest) =
queryOperation(tableSpec).execute(request)
override suspend fun scan(request: ScanRequest) =
scanOperation(tableSpec).execute(request)
}
```
Known work remaining
Several TODOs and FIXMEs are to be found right now. At the very least I know we'll need to:
Replace function-style builders (e.g., for request/response creation) with DSL-style builders (similar to what we use in the low-level clients)
Render DSL-style extension methods for operations (similar to what we do for low-level operations)
Give KSP clearer indications of which output files depend on which input files. Right now, KSP can't detect which updates should trigger re-generating which code so it generates all the code on every build.
Issue \
Part of https://github.com/awslabs/aws-sdk-kotlin/issues/76
Description of changes
At long last, the wait for codegenned mapper operations and high-level types is ~done~ started! This change removes the handwritten toy implementation of a
getItem
operation and now generates the operation glue code for the following DDB ops:deleteItem
getItem
putItem
query
*scan
** These operations should be paginated but the codegen doesn't handle that yet. It's left as a
FIXME
in the code for follow-up.PR flight plan
⚠️ There's a lot to take in here, especially if you're unfamiliar with KSP. I recommend taking a short break to learn some basics about KSP before getting started with the review.
Recommended order of review:
HighLevelOpsProcessor
which is the main entry point for KSP. Everything else in this module is called somehow from this point.model
package is for data types for describing the low- & high-level operations, structures, and typescore
package contains rudimentary code writing utilities such as indentation, blocks, imports, and template processing. These are good candidates for commonizing somewhere in the future, since we'll likely need similar functionality for DDB mapper annotation processing and possibly other HLLs in the future.rendering
package contains structured codegen for rendering the operations, convenience functions, and typesoperations
package which use DynamoDB Local in lieu of mocking or making actual calls to DDBSample codegen output
Understanding the codegen may be easier if you see the generated code. All code is generated into the hll/ddb-mapper/dynamodb-mapper/build/generated/ksp/common/commonMain/kotlin directory.
GetItem operation
```kotlin // Code generated by ddb-mapper-ops-codegen. DO NOT EDIT! package aws.sdk.kotlin.hll.dynamodbmapper.operations import aws.sdk.kotlin.hll.dynamodbmapper.TableSpec import aws.sdk.kotlin.hll.dynamodbmapper.items.ItemSchema import aws.sdk.kotlin.hll.dynamodbmapper.model.toItem import aws.sdk.kotlin.hll.dynamodbmapper.pipeline.internal.HReqContextImpl import aws.sdk.kotlin.hll.dynamodbmapper.pipeline.internal.MapperContextImpl import aws.sdk.kotlin.hll.dynamodbmapper.pipeline.internal.Operation import aws.sdk.kotlin.services.dynamodb.model.ConsumedCapacity import aws.sdk.kotlin.services.dynamodb.model.ReturnConsumedCapacity import kotlin.Boolean import kotlin.String import aws.sdk.kotlin.services.dynamodb.model.GetItemRequest as LowLevelGetItemRequest import aws.sdk.kotlin.services.dynamodb.model.GetItemResponse as LowLevelGetItemResponse public interface GetItemRequestTable operations interface/implementation
```kotlin // Code generated by ddb-mapper-ops-codegen. DO NOT EDIT! package aws.sdk.kotlin.hll.dynamodbmapper.operations import aws.sdk.kotlin.hll.dynamodbmapper.TableSpec /** * Provides access to operations on a particular table, which will invoke low-level operations after * mapping objects to items and vice versa * @param T The type of objects which will be read from and/or written to this table */ public interface TableOperationsKnown work remaining
Several
TODO
s andFIXME
s are to be found right now. At the very least I know we'll need to:Query
,Scan
, etc.) correctlyBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.