search

feast-dev / feast

The Open Source Feature Store for Machine Learning
https://feast.dev
Apache License 2.0
5.51k stars 982 forks source link

Add `online_read_async` method to Postgres Online Store #4260

Closed job-almekinders closed 2 months ago

job-almekinders commented 3 months ago

Is your feature request related to a problem? Please describe. With the implementation of get_online_features_async to the FeatureStore class (Issue and PR), online_read_async methods are required for each online store in order to be able to retrieve features in an async manner. This issue aims to implement this for the Postgres Online Store.

Describe the solution you'd like Add a online_read_async method to Postgres Online Store.

However, the current Postgres implementation within feast leverages psycopg2, which doesn't support async operations. The newer version - psycopg3 - does however support this.

Therefore, we would like to propose to implement this change with two separate PRs:

  1. A PR which upgrades the psycopg2 dependency to use psycopg3. The latter tries to behave as close as possible to the former, however, there are a few differences that need to be accounted for. A full list of difference can be found here.
  2. A PR which implements the online_read_async method on the Postgres Online Store.

Would love to hear your thoughts on this approach before I start working on this!

Describe alternatives you've considered Another library to handle async postgres operations is asyncpg. However, this repos latest release was 5 Nov, 2023. Furthermore, the last commit was three months ago, and the last 7 months only 5 commits are made in total. Since this seems quite limited, we don't think it would be a good move to add this library to the dependencies for the postgres extension within feast.

tokoko commented 3 months ago

sounds good to me. I'd go with psycopg3 as well. One thing to note is that postgres offline store also depends on psycopg2. I guess we can have both as dependencies as they're different libraries, but maybe we should look into upgrading offline store as well, if it's not too complicated to make a switch. wdyt?

job-almekinders commented 3 months ago

I think that makes sense!

Once I start with the implementation I'll see how much effort it is and whether we can pick it up in one go 👍