search

singer-io / tap-google-sheets

GNU Affero General Public License v3.0
34 stars 48 forks source link

How to select stream from where the data will be retrieved #8

Open davicorreiajr opened 4 years ago

davicorreiajr commented 4 years ago

Running

tap-google-sheets --config config.json --discover > catalog.json

Gives the catalog.json file, as expected (with schema, stream name, metadata, etc).

But now, I want to actually get data from one sheet (say, "Sheet 1"). How am I supposed to do that? Because running:

tap-google-sheets --config config.json --catalog catalog.json

gives me nothing.

So I tried to understand what was happening, and found this: https://github.com/singer-io/singer-python/blob/6c6c773d8b6dc6223551e598574eb0df41f0c415/singer/catalog.py#L47, which basically verifies if a stream is selected. But it turns out this is not automatically generated in the catalog file; so I needed to go to the specific stream ("Sheet 1") and add "selected": true inside the schema property.

Am I missing something here? I think it should be a way to automatically select which stream ("Sheet 1", "file_metadata", etc) you want to get data from.

GAZ082 commented 4 years ago

That's pretty much what the docs says:

https://github.com/singer-io/getting-started/blob/master/docs/DISCOVERY_MODE.md#metadata