search

meltano / sdk

Write 70% less code by using the SDK to build custom extractors and loaders that adhere to the Singer standard: https://sdk.meltano.com
https://sdk.meltano.com
Apache License 2.0
94 stars 69 forks source link

known limitation: "Stream Maps" not currently compatible with `BATCH` messages #1117

Open aaronsteers opened 1 year ago

aaronsteers commented 1 year ago

The initial support for BATCH message types is currently incompatible with the Stream Maps feature.

In theory, this could definitely be added - although at the cost of performance.

Related:

stale[bot] commented 1 year ago

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

ReubenFrankel commented 3 weeks ago

Would you accept a PR to partially implement stream aliasing via stream maps (__alias__) for BATCH messages? https://github.com/Matatika/tap-bigquery/commit/9c96860aaa1439c1560a4ca62a0beb2040c1f368

edgarrmondragon commented 3 weeks ago

Would you accept a PR to partially implement stream aliasing via stream maps (__alias__) for BATCH messages? https://github.com/Matatika/tap-bigquery/commit/9c96860aaa1439c1560a4ca62a0beb2040c1f368

Go for it! That's something we can reasonably support.