search

MeltanoLabs / tap-postgres

Singer Tap for PostgreSQL
https://hub.meltano.com/extractors/tap-postgres--meltanolabs/
Other
19 stars 23 forks source link

Parquet type batch config, pyarrow error #361

Open resulyrt93 opened 7 months ago

resulyrt93 commented 7 months ago

Hey team When I try parquet type batch config, it raises error like ModuleNotFoundError: No module named 'pyarrow'. I checked your plugin dependencies, in here pyarrow configured as extra. However I can't figure out that how set it with extra config on meltano.

I've tried following config with reference that comment but it didn't work.

pip_url: "meltanolabs-tap-postgres[parquet]@git+https://github.com/MeltanoLabs/tap-postgres"
edgarrmondragon commented 7 months ago

@resulyrt93 currently parquet is only an extra of the singer-sdk package, so you'd need something like

pip_url: "'meltanolabs-tap-postgres @ git+https://github.com/MeltanoLabs/tap-postgres' singer-sdk[parquet]"

FWIW we could also make parquet an extra of this package, so PRs welcome!