MeltanoLabs / meltano-map-transform

A map transformer which implements the `Stream Maps` capability from Meltano's tap and target SDK: https://sdk.meltano.com/
Apache License 2.0
18 stars 15 forks source link

Accept a series of one or more regex replacement rules for stream aliasing #26

Open aaronsteers opened 2 years ago

aaronsteers commented 2 years ago

As of now, stream name aliasing is performed one stream at a time. This doesn't scale very well for use cases where we actually want to apply a naming convention or standard 'morph' of stream names between tap and target.

As raised in slack: https://meltano.slack.com/archives/C013EKWA2Q1/p1656091898668519?thread_ts=1656084258.386049&cid=C013EKWA2Q1

Regex approach

With a global regex substitution rule like the camelcase-to-snakecase stack overflow example and demoed here in regex101,

we could convert names like

myTableName
myTableName2
MyTableName3
ThisIsALineOfText87

into names like

MY_TABLE_NAME
MY_TABLE_NAME_2
MY_TABLE_NAME_3
THIS_IS_A_LINE_OF_TEXT_87

which is a common requirement for landing data into Snowflake.

yjw868 commented 2 years ago

Hello @aaronsteers, is there any update on this? Thanks.