feast-dev / feast

The Open Source Feature Store for Machine Learning
https://feast.dev
Apache License 2.0
5.63k stars 1k forks source link

feat: feast teardown command brings all projects down regardless of which feature_repo you are in in shared registry #3591

Open jmeisele opened 1 year ago

jmeisele commented 1 year ago

Expected Behavior

feast teardown command in the feature_repo will remove resources only tied to the project listed in the corresponding feature_store.yaml

Current Behavior

feast teardown command in any feature repo referencing the same registry removes all projects resources under that registry

Steps to reproduce

You can use the same GItHub repo or spread across multiple repos.

  1. Create feature repo 1

    • feast init -t snowflake snowflake_feast
    • Fill out generated snowflake cred prompts in CLI
    • cd snowflake_feast/feature_repo
    • feast apply
  2. Create feature repo 2

    • feast init -t snowflake concrete_opossum
    • Fill out generated snowflake cred prompts in CLI
    • cd concrete_opossum/feature_repo
    • feast apply

You'll see both projects listed in registry metadata (as expected).

image

  1. Perform feast teardown in either feature_repo directory we generated earlier

Specifications

Possible Solution

feast teardown-project command only removes resources tied to the project listed in the feature_repo/feature_store.yaml it is run from and leaves the other projects infra alone in the registry.

jmeisele commented 1 year ago

I re-read the docs to see if there's something I'm missing using the SQL based registry (Snowflake). Bullet 2 references tearing down a project (desired behavior I'm after). I didn't see a CLI command to teardown_project or anything similar. The teardown command does indeed remove the registry.

image

wjhhuizi commented 1 year ago

I don't think the multi-project implementation for the FEAST working really well in the registry at this moment... there need some code works for that part...

sfc-gh-madkins commented 1 year ago

@jmeisele can you retag this as an improvement? given this is currently not a bug

jmeisele commented 1 year ago

@sfc-gh-madkins Tried to change the label from GitHub and I cannot, can you from your side?

sfc-gh-madkins commented 1 year ago

I got it ... @jmeisele let me know how big of a blocker this is for you

jmeisele commented 1 year ago

@sfc-gh-madkins we can't adopt feast until we can spin up/spin down infra on a project by project basis šŸ˜­ so the sooner we can get this capability released the sooner we can start using Feast. More than happy to contribute šŸš€

sfc-gh-madkins commented 1 year ago

@jmeisele want to look at some code for this?

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.