meltano / sdk

Write 70% less code by using the SDK to build custom extractors and loaders that adhere to the Singer standard: https://sdk.meltano.com
https://sdk.meltano.com
Apache License 2.0
94 stars 67 forks source link

refactor: SQLConnector methods that take `full_table_name` should take an object with pre-parsed table name parts #1177

Open edgarrmondragon opened 1 year ago

edgarrmondragon commented 1 year ago

Feature scope

Targets (data type handling, batching, SQL object generation, etc.)

Description

The SQLConnector has at the moment of writing 12 methods that take full_table_name as argument and this string has to be parsed multiple times into its constituent parts.

There is a (probably small) performance penalty for doing that, and it' a less-than-ideal API for target developers (for example, see https://github.com/meltano/sdk/issues/1176).

BuzzCutNorman commented 1 year ago

@edgarrmondragon I completely agree with what you said but I don't know if the issue you referenced is the best example. Looking more into the example issue it seems to be a related to the conform_name more than splitting full_table_name. wdyt?

edgarrmondragon commented 1 year ago

@edgarrmondragon I completely agree with what you said but I don't know if the issue you referenced is the best example. Looking more into the example issue it seems to be a related to the conform_name more than splitting full_table_name. wdyt?

@BuzzCutNorman Yeah, I could've been more clear. I meant to mention both conformed column names, as well as parsed table names. The former will be called at least twice to fix the issue as the API stands, the latter is already called multiple times. That's why it's a good idea to perform that operation once and pass a wrapper around instead of the raw values.

stale[bot] commented 1 year ago

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

edgarrmondragon commented 1 year ago

Still relevant