Blog post: Using Custom Python connectors with PyAirbyte

bindipankhudi commented 1 month ago

Summary

This blog post would focus on how to build custom python connectors and then executing them from PyAirbyte.

Definition of Done

Blog post illustrating how to build custom python connectors and then executing them from PyAirbyte.

Article should cover:

Basic instruction on how to start building and testing a Python-based source connector.
How to install and test a Python connector in PyAirbyte using pip_url of local path.
Show example for installing a Python connector in PyAirbyte using pip_url of a git repo and/or with the pip_url of a git feature branch.
Any suggestions for debugging / troubleshooting which seem relevant for this use case.

Resources to Assist

PyAirbyte tutorials: https://github.com/airbytehq/quickstarts/tree/main/pyairbyte_notebooks
PyAirbyte Documentaton: https://docs.airbyte.com/using-airbyte/pyairbyte/getting-started
PyAirbyte API Reference: https://docs.airbyte.com/using-airbyte/pyairbyte/getting-started#api-reference
Sample blog post: https://airbyte.com/blog/manage-and-orchestrate-airbyte-connections-using-python

avirajsingh7 commented 1 month ago

@aaronsteers i think both this issue are almost same https://github.com/airbytehq/PyAirbyte-Hackathon/issues/7

aaronsteers commented 4 weeks ago

@avirajsingh7 - Apologies for the confusion. I've updated this issue's description and title to target Python connectors, specifically, focusing on local testing and how to run from feature branches during development. I've also clarified the similar item to focus more clearly on no-code developer, which outputs the connector definition as a yaml file.

Let me know if this is still interesting - and if so, let me know if I can provide any additional resources to assist or clarify. Thanks!

marcosmarxm commented 4 weeks ago

@topefolorunso do you want to work on this one?

Harmaton commented 3 weeks ago

Assign this to me @marcosmarxm

marcosmarxm commented 2 weeks ago

@Harmaton do you need any help here?

Harmaton commented 2 weeks ago

@marcosmarxm I have this doc ready. Could you please review ? @bindipankhudi

Harmaton commented 2 weeks ago

If done. can you assign me issue #19 @marcosmarxm

aaronsteers commented 1 week ago

I completed a review and it looks good for my part. I'll let @marcosmarxm handle next steps during his working hours tomorrow. (Today is a US / company holiday.)

Harmaton commented 1 week ago

@marcosmarxm

marcosmarxm commented 1 week ago

@Harmaton you can proceed to the next challenge. I'll post this in Airbyte blog.

Harmaton commented 1 week ago

Please mark as done while at it. I know You have my bio. Thank you!

airbytehq / PyAirbyte