search

feast-dev / feast

The Open Source Feature Store for Machine Learning
https://feast.dev
Apache License 2.0
5.58k stars 996 forks source link

Semi-isolated subprojects #4199

Open tokoko opened 5 months ago

tokoko commented 5 months ago

Is your feature request related to a problem? Please describe.

Feast has a concept of a project that can be used to host several completely isolated resources in a shared registry. This can be used to co-host registry objects for unrelated teams or environments like dev, test, prod. (honestly, not sure why would anybody do that though, a separate registry for each sounds a lot better, there's not much of an overhead). This doesn't address the use case when a user might want to host a shared pool of resources with various ownerships, for example some core objects (entities) that are managed by a core team, with additional feature views and feature services that are owned by different teams.

This is hard to achieve with normal feast apply workflow as applying a subset of feast objects will lead to the removal of other objects.

Describe the solution you'd like

  1. Introduce a concept of a subproject (or maybe call this something else, happy to consider alternatives) which will be used to mark feast objects that are managed as a whole from a single git repository. The user should be able to run feast apply --subproject team-1 that will overwrite only existing objects that were part of this subproject. Any attempt to apply a change that will break the integrity of a registry as a whole (trying to remove an entity that's referenced by a feature view that belongs to another subproject) should lead to an error.

  2. Another option is to repurpose the existing project label for this, but this might be too big of a breaking change.

P.S. Subprojects should eventually also fit into the overall security model #4198.

tokoko commented 2 months ago

@EXPEbdodla curious what you think about this

EXPEbdodla commented 2 months ago

@tokoko Glad you tagged me to this. I was thinking this as an extension but slightly different. What I'm thinking is, Project object should store RepoConfig object instead of defining feature_store.yaml. We can still use feature_store.yaml as an override. feature_store.yaml can help to override multiple projects if necessary.

All the Feast Proto objects already have Project name in their spec. We need to update Feast Objects in Python to accept Project name So we can tag Feast Objects to Projects.

Lets say we have a single GITHUB repo to manage all the projects.

During Registration/ Apply stage, We apply all the projects. GITHUB Repo will be in sync with registry for the specified projects. or we can only apply projects specified.

During Materialization stage, Materialize the feature views based on project name and feature view name specified. RepoConfig is already registered in Registry. We can use that or accept override.

During Retrieval from Online Store, if the projects share same infra/ online store, then they can spin up feature server with multiple projects. Currently it's one feature server per project. It helps to reduce the operational burden in maintaining multiple feature servers. We may need to introduce project as an option in the request or enforce unique feature view names across all the projects during Apply stage to avoid conflicts.

This should definitely work with what you want to achieve with environments and also for multiple teams under the same org. I'm not sure if any wants to have different Feast objects per environment. This may not be a good practice because the objects defined are not tested in lower environments before proceeding to the production environments. If we want to avoid that, during apply, we can stop supporting Apply at project level.

Hope this aligns with what you and other maintainers are envisioning. Thank you.

EXPEbdodla commented 2 months ago

Without Repoconfig object in Project proto schema, we can treat all the configuration in feature_store.yaml as default for all projects.

tokoko commented 2 months ago

A couple of thoughts and questions.

What I'm thinking is, Project object should store RepoConfig object instead of defining feature_store.yaml. We can still use feature_store.yaml as an override. feature_store.yaml can help to override multiple projects if necessary.

I'm not sure about this one. RepoConfig has been used lately, especially with the addition of remote deployment options, as more of a client configuration object. Storing client config in a server doesn't feel right, although I see that it might make onboarding users easier.

This should definitely work with what you want to achieve with environments and also for multiple teams under the same org.

I think the part where your outlook differs from my proposed solution is that even if we manage to simplify managing multiple projects under central infrastructure, project concept in feast is still assumed to be a completely isolated set of resources (from the perspective of actual usage, not infra). For use cases where different teams need to author different feature resources in a shared project, we will still need to provide tools to support that, right?

if the projects share same infra/ online store, then they can spin up feature server with multiple projects.

As long as the requirement of "if the projects share same infra/ online store" is there, wouldn't it be simpler to accomplish this by converging all these resources in a single project and controlling access with rbac?