feast-dev / feast

The Open Source Feature Store for Machine Learning
https://feast.dev
Apache License 2.0
5.62k stars 1k forks source link

Fetching historical feature values by passing sql as entity_df #3234

Open TaherehFarrokhi opened 2 years ago

TaherehFarrokhi commented 2 years ago

Expected Behavior

In Feast version 0.18.1, it was poosible to get historical features by passing a sql query as entity_df and it returns the values

Current Behavior

When I updated or .24+ it returns error : string entities are currently not supported in the MsSQL offline store.

https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/offline_stores/contrib/mssql_offline_store/mssql.py#L189

Steps to reproduce

this is my feast_store.yaml

registry:
  registry_store_type: AzureRegistryStore
  path: ${REGISTRY_PATH}
project: FinalUniverseY1
provider: azure
online_store:
    type: redis
    connection_string: ${REDIS_CONN}
offline_store:
    type: mssql
    connection_string: ${SQL_CONN}
  1. Defined entity
  2. define feature view
  3. Write this code:
    feature_view = self.feature_store.get_feature_view(fq.feature_view_name)
    entity = self.feature_store.get_entity(feature_view.entities[0])
    features = [f"{feature_view.name}:{f.name}" for f in feature_view.features]
      batch_source: MsSqlServerSource = feature_view.batch_source
    entity_df = f"select {entity.join_key}, {feature_view.batch_source.timestamp_field} as event_timestamp from [dbo].{batch_source.table_ref} where {batch_source.timestamp_field} between '{fq.from_date}' and '{fq.to_date}'"
      feature_service = self.feature_store.get_feature_service("test_service_1")
    job = self.feature_store.get_historical_features(
          entity_df=entity_df, features=feature_service
      )
    df = job.to_df()

Specifications

Possible Solution

Nevermind-s commented 1 year ago

Any news on this one?

amommendes commented 1 year ago

Seems to me this is an implementation decision and not a bug (whether introduced or not), then this could be a feature request... The old version ran a SELECT INTO from sql string in the entity table

tsenhungwu commented 1 year ago

I encountered a similar issue where the insertion operation defaults the string column in an entity table with varchar(1) which violates the length check.

adchia commented 1 year ago

I believe this might have been just an incorrect gate. The other offline stores allow for this and it seemed to work in the old version. We should be able to re-enable this and fix any tests that are failing as a result

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.