search

airbytehq / PyAirbyte-Hackathon

Tasks for PyAirbyte Hackathon June 2024
0 stars 2 forks source link

Create new PGVector Airbyte destination #5

Closed aaronsteers closed 1 month ago

aaronsteers commented 6 months ago

Summary

Modeling after the Snowflake Cortex destination, this new connector would support loading to Postgres "PGVector" as a local vector store native to Postgres.

Project Description

Definition of Done

Resources to Assist

Hashcode-Ankit commented 5 months ago

@aaronsteers this is my second choice first one is this one: https://github.com/airbytehq/PyAirbyte-Hackathon/issues/6

marcosmarxm commented 5 months ago

@Hashcode-Ankit I assigned another issue to you.

bala-ceg commented 5 months ago

Hi Aaron,Marcos Ideally to develop this Vector DB destination, is there any samples that i can refer?

marcosmarxm commented 5 months ago

@aaronsteers do you have any resources?

aaronsteers commented 5 months ago

Hi, @bala-ceg . The best resource right now would be to start with a copy of the new Snowflake Cortex destination. That can be found in the airbytehq/airbyte repo under airbyte-integrations/connectors/destination-snowflake-cortex.

This destination combines helpers from PyAirbite as well as from the vector store CDK, and is probably the best place to start.

Basic Postgres functionally can be relied upon from the PosgresSqlProcessor class in PyAirbyte, similar to how that destination builds on the Snowflake processor.

Hope this helps!!

fvgm-spec commented 4 months ago

Hi @marcosmarxm could you please assign this one to me?

aaronsteers commented 4 months ago

@fvgm-spec - It's yours! 🙏 Let us know if you have any questions!

fvgm-spec commented 4 months ago

Hi @aaronsteers , could you please unassign me this issue? As I didn't have enough time to work on it. Thanks!

aaronsteers commented 1 month ago

Shipped by @aldogonzalez8 last week! 🚀 🙌

Very excited to see this launched.

Closing this hackathon issue now that the connector is live.