search

singer-io / singer-python

Writes the Singer format from Python
https://singer.io
Apache License 2.0
537 stars 129 forks source link

--catalog vs -p (properties) parameters #70

Open chrisb-c01 opened 6 years ago

chrisb-c01 commented 6 years ago

It might be me, but the use of the --catalog and property -p parameters is somewhat unclear to me.

They are used intermixed throughout different sections of the docs (e.g. Allowing Users to Select Streams to Sync vs. Sync mode).

Source code mentions -p to be deprecated over --catalog, however replacing -p by --catalog leads to different behaviour: the "selected": true on a schema definition in the catalog file is not honoured by --catalog, but is by -p parameter.

It would be helpful if this could be clarified a bit more in the docs.

briansloane commented 6 years ago

Thanks for the feedback, @ChrisCalculus. As you noted, the -p- is deprecated in favor of using the --catalog. We're shifting towards usage of metadata in the catalog for noting which fields should be selected. We agree that the documentation needs some work and plan to address that in the future. We'll update here when that happens.

chrisb-c01 commented 6 years ago

Thanks for replying @briansloane. Looking forward to the update.

pedromachados commented 6 years ago

Any plans to change the transform function to respect the selected setting?

briansloane commented 6 years ago

@pedromachados No plans to do that as of now. I'd recommend creating another issue to outline this request