Open am29d opened 9 months ago
Copying this comment I have left under a PR since it's relevant to the conversation:
[...] I think we should agree & articulate our stance on this topic a bit better to avoid future confusion amongst us and towards customers.
The main point here, besides calling out potential discrepacies between us & the other library, is in my opinion to establish our strategy towards event types as a whole. This will then influence which actions we'll take now and in the future, as well as when discrepacies are detected.
Currently the @types/aws-lambda
types are community maintained and manually authored. AWS events can change and the types can contain issues.
While I agree with the sentiment of maintaining these types and our schemas (and in the future JSON schemas as well) up to date and aligned, I don't think this ultimately guarantees that the types/schemas are actually correct.
As a think big idea, we should at least consider the option of instead investing efforts in creating automations to periodically test against real AWS events to get the actual shapes of the events sent to Lambda. Based of those, we could then derive the schemas as well as the TypeScript types.
Once we detect drift (aka changes in the events), we could then propagate them to the appropriate destinations be it within the scope of Powertools or @types/aws-lambda
.
I am not expecting the whole industry to move over our own types (i.e. @aws-lambda-powertools/events
or @aws-lambda-powertools/types
) but I would still aim at becoming the source of truth for these events/schemas/types.
The DefinitelyTyped repo (aka the one that hosts @types/*
packages) already has precedents of depending on external type definition, one of them was added in the past 6 months to the @types/node
types to support the fetch
module. This module was originally implemented under undici
and later incorporated into Node.js. The source still resides in the undici
repo, so to add support for these types they started publishing type definitions for under undici-types
that are now a direct depenencey pf the @types/node
package (see here).
While it'll be a lot of work, I believe this would bring a lot of value to customers and eventually would help with establishing a single and authoritative source of truth for AWS events that today does not exist. Doing this would benefit not only Powertools TS or the Node.js community but also Powertools Python and the Python community as a whole via Pydantic. Additionally, it would open the door to future use cases like testing factories (#445), integrations with SAM or CDK, and similar.
Summary
When using our built-in schema with zod, developers can infer the types with
type SqsEvent = z.infer<typeof SqsSchema>
and use it in their Lambda handler. Another popular option is to use@types/aws-lambda
package wich has 3M downloads per week. We need to make sure that both types are compatible, so there is no confusion or unexpected behaviour.Why is this needed?
If either
@types/aws-lambda
or our schema is updated, we want to have a reliable test to verify that they are both compatible and if not to fix one them.Which area does this relate to?
Parser
Solution
I have checked the first idea that worked:
The compiler will fail if the structure does not match and will tell exactly which properties are not compatible.
Alternatively research more about https://github.com/fabien0102/ts-to-zod. If we can convert
@types/aws-lambda
to schemas, we might directly compare the schemas to find the differences.Acknowledgment
Future readers
Please react with 👍 and your use case to help us understand customer demand.