Open lngvietthang opened 1 day ago
0.41.0
3.9
Mapping (stream maps, flattening, etc.)
Linux
In my Meltano configuration, I use meltano-map-transformer (latest version with singer-sdk@0.41.0) with config as below:
stream_maps: xxx: creation: datetime.datetime.strptime(creation, '%Y-%m-%dT%H:%M:%S.%f%z') ...
With new update from singer-sdk v0.41, this field (creation) is mapped with date type in schema message.
creation
date
class CustomStreamMap(StreamMap): def _eval_type( # noqa: PLR0911 self, expr: str, default: th.JSONTypeHelper | None = None, ) -> th.JSONTypeHelper: ... if expr.startswith("datetime.date") or expr.endswith(".date()"): return th.DateType() if expr.startswith("datetime.datetime"): return th.DateTimeType() ....
I think we need to swap the position of 2 if clause to check the datetime data type firstly.
if
datetime
Related PR:
class CustomStreamMap(StreamMap): def _eval_type( # noqa: PLR0911 self, expr: str, default: th.JSONTypeHelper | None = None, ) -> th.JSONTypeHelper: ... if expr.startswith("datetime.date") or expr.endswith(".date()"): return th.DateType()
if expr.startswith("datetime.datetime"): return th.DateTimeType() ....
No response
That makes sense @lngvietthang. PRs welcome!
Singer SDK Version
0.41.0
Is this a regression?
Python Version
3.9
Bug scope
Mapping (stream maps, flattening, etc.)
Operating System
Linux
Description
In my Meltano configuration, I use meltano-map-transformer (latest version with singer-sdk@0.41.0) with config as below:
With new update from singer-sdk v0.41, this field (
creation
) is mapped withdate
type in schema message.I think we need to swap the position of 2
if
clause to check thedatetime
data type firstly.Related PR:
2665
Code
class CustomStreamMap(StreamMap): def _eval_type( # noqa: PLR0911 self, expr: str, default: th.JSONTypeHelper | None = None, ) -> th.JSONTypeHelper: ... if expr.startswith("datetime.date") or expr.endswith(".date()"): return th.DateType()
Link to Slack/Linen
No response