search

feast-dev / feast

The Open Source Feature Store for Machine Learning
https://feast.dev
Apache License 2.0
5.48k stars 978 forks source link

Can't import custom offline store #3883

Closed killkamad closed 8 months ago

killkamad commented 8 months ago

Expected Behavior

Import and use custom offline store. Please show me how to properly import Custom Offline Store.

Current Behavior

I get this error when trying to use Feast:

2024-01-12 11:28:46 /usr/local/lib/python3.9/site-packages/dask/dataframe/_pyarrow_compat.py:23: UserWarning: You are using pyarrow version 11.0.0 which is known to be insecure. See https://www.cve.org/CVERecord?id=CVE-2023-47248 for further details. Please upgrade to pyarrow>=14.0.1 or install pyarrow-hotfix to patch your current version.
2024-01-12 11:28:46   warnings.warn(
2024-01-12 11:28:46 Traceback (most recent call last):
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/importer.py", line 26, in import_class
2024-01-12 11:28:46     module = importlib.import_module(module_name)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
2024-01-12 11:28:46     return _bootstrap._gcd_import(name[level:], package, level)
2024-01-12 11:28:46   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
2024-01-12 11:28:46   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
2024-01-12 11:28:46   File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
2024-01-12 11:28:46   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
2024-01-12 11:28:46   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
2024-01-12 11:28:46   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
2024-01-12 11:28:46   File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
2024-01-12 11:28:46 ModuleNotFoundError: No module named 'feast_custom_offline_store'
2024-01-12 11:28:46 
2024-01-12 11:28:46 The above exception was the direct cause of the following exception:
2024-01-12 11:28:46 
2024-01-12 11:28:46 Traceback (most recent call last):
2024-01-12 11:28:46   File "/usr/local/bin/feast", line 8, in <module>
2024-01-12 11:28:46     sys.exit(cli())
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
2024-01-12 11:28:46     return self.main(*args, **kwargs)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1078, in main
2024-01-12 11:28:46     rv = self.invoke(ctx)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
2024-01-12 11:28:46     return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
2024-01-12 11:28:46     return ctx.invoke(self.callback, **ctx.params)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
2024-01-12 11:28:46     return __callback(*args, **kwargs)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
2024-01-12 11:28:46     return f(get_current_context(), *args, **kwargs)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/cli.py", line 170, in ui
2024-01-12 11:28:46     store = create_feature_store(ctx)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/usage.py", line 299, in wrapper
2024-01-12 11:28:46     raise exc.with_traceback(traceback)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/usage.py", line 288, in wrapper
2024-01-12 11:28:46     return func(*args, **kwargs)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/repo_operations.py", line 351, in create_feature_store
2024-01-12 11:28:46     return FeatureStore(repo_path=str(repo), fs_yaml_file=fs_yaml_file)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/usage.py", line 342, in wrapper
2024-01-12 11:28:46     return func(*args, **kwargs)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/feature_store.py", line 152, in __init__
2024-01-12 11:28:46     self.config = load_repo_config(self.repo_path, fs_yaml_file)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/repo_config.py", line 549, in load_repo_config
2024-01-12 11:28:46     c = RepoConfig(**raw_config)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/repo_config.py", line 185, in __init__
2024-01-12 11:28:46     super().__init__(**data)
2024-01-12 11:28:46   File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
2024-01-12 11:28:46   File "pydantic/main.py", line 1050, in pydantic.main.validate_model
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/repo_config.py", line 385, in _validate_offline_store_config
2024-01-12 11:28:46     offline_config_class = get_offline_config_from_type(offline_store_type)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/repo_config.py", line 530, in get_offline_config_from_type
2024-01-12 11:28:46     return import_class(module_name, config_class_name, config_class_name)
2024-01-12 11:28:46   File "/usr/local/lib/python3.9/site-packages/feast/importer.py", line 31, in import_class
2024-01-12 11:28:46     raise FeastModuleImportError(module_name, class_name) from e
2024-01-12 11:28:46 feast.errors.FeastModuleImportError: Could not import module 'feast_custom_offline_store.file' while attempting to load class 'CustomFileOfflineStoreConfig'

Steps to reproduce

I tried to run my own offline store but got the error as above. And also tried to use this repository https://github.com/feast-dev/feast-custom-offline-store-demo and again got the error on my local "Docker Desktop" environment as well as on the “Debian GNU/Linux 10” server.

Specifications

killkamad commented 8 months ago

I found a solution, you have to add the PYTHONPATH to the path where my dir with module is located in Dockerfile, like this:

ENV PYTHONPATH "${PYTHONPATH}:/app"

My module is located in feast_oracle dir. Example: