aimementoring / tap-airtable

Singer TAP for Airtbale
8 stars 26 forks source link

Enable discovery mode #4

Closed bfischetti closed 4 years ago

bfischetti commented 5 years ago

This is already done in the tap but in a different way. We only need to adapt the way we call this method (right now everything is on the run_taps.py).

Discovery mode provides a way for a tap to describe the data streams it supports. JSON schema is used to describe the structure and type of data for each stream. The implementation of discovery mode will depend on the tap's data source. Some taps will hard code the schema for each stream, while others will connect to an API that provides a desription of the available streams. When discovery mode is run, the tap should write to stdout a list of streams, known as the catalog, with each entry containing some basic information about the stream and a JSON schema describing the stream's data.

To run a tap in discovery mode, the --discover flag should be provided:

tap --config CONFIG --discover Discovery is typically run with the output redirected to a file so it can be passed into the tap in sync mode:

tap --config CONFIG --discover > catalog.json

Sub-task of #1

bfischetti commented 4 years ago

PR opened for this: #7

bfischetti commented 4 years ago

Done!