Adds a AirbyteConnection(JobBlock) and AirbyteSync(JobRun) enabling the new flow run_connection_sync.
Adds a new client method get_job_info to allow inclusion of records_synced in the AirbyteSyncResult.
Closes #22
Example
from prefect import flow
from prefect_airbyte.server import AirbyteServer
from prefect_airbyte.connections import AirbyteConnection
from prefect_airbyte.flows import run_connection_sync
server = AirbyteServer()
connection = AirbyteConnection(
airbyte_server=server,
connection_id="7e78757d-fd7f-421b-80a5-f329dbdadf1b",
status_updates=True,
)
@flow
def airbyte_syncs():
# do some setup
sync_result = run_connection_sync(
airbyte_connection=connection,
)
# do some other things, like trigger DBT based on number of records synced
print(f'{sync_result.records_synced}')
Screenshots
Checklist
[x] References any related issue by including "Closes #" or "Closes ".
If no issue exists and your change is not a small fix, please create an issue first.
[x] Includes tests or only affects documentation.
[x] Passes pre-commit checks.
Run pre-commit install && pre-commit run --all locally for formatting and linting.
[x] Includes screenshots of documentation updates.
Adds a
AirbyteConnection(JobBlock)
andAirbyteSync(JobRun)
enabling the new flowrun_connection_sync
.Adds a new client method
get_job_info
to allow inclusion ofrecords_synced
in theAirbyteSyncResult
.Closes #22
Example
Screenshots
Checklist
pre-commit
checks.pre-commit install && pre-commit run --all
locally for formatting and linting.mkdocs serve
view documentation locally.