search

great-expectations / great_expectations

Always know what to expect from your data.
https://docs.greatexpectations.io/
Apache License 2.0
9.71k stars 1.5k forks source link

circular import error on great_expectations.core.usage_statistics.usage_statistics #8140

Open ralemy opened 1 year ago

ralemy commented 1 year ago

Describe the bug Import of BaseDataConfig fails due to circular import error in usage_statistics module To Reproduce `

pip show great_expectations Name: great-expectations Version: 0.17.0 Summary: Always know what to expect from your data. Home-page: https://greatexpectations.io Author: The Great Expectations Team`

simply importing BaseDataContext:

from great_expectations.data_context import BaseDataContext

Fails with Import Error:

`ImportError Traceback (most recent call last) Cell In[34], line 8 1 # Great Expectations: 2 3 (...) 6 # import great_expectations as ge 7 # from great_expectations.data_context.types.base import DataContextConfig, FilesystemStoreBackendDefaults, DatasourceConfig ----> 8 from great_expectations.data_context import BaseDataContext 9 # def prepare_ge_context(root_dir: str) -> BaseDataContext: 10 # data_context_config = DataContextConfig( 11 # store_backend_defaults=FilesystemStoreBackendDefaults( (...) 15 # ge_context = BaseDataContext(project_config=data_context_config) 16 # return ge_context

File /opt/conda/lib/python3.11/site-packages/great_expectations/init.py:6 2 from ._version import get_versions # isort:skip 4 version = get_versions()["version"] # isort:skip ----> 6 from great_expectations.data_context.migrator.cloud_migrator import CloudMigrator 7 from great_expectations.expectations.registry import register_core_expectations 9 del get_versions # isort:skip

File /opt/conda/lib/python3.11/site-packages/great_expectations/data_context/init.py:1 ----> 1 from great_expectations.data_context.data_context import ( 2 AbstractDataContext, 3 BaseDataContext, 4 CloudDataContext, 5 DataContext, 6 EphemeralDataContext, 7 FileDataContext, 8 )

File /opt/conda/lib/python3.11/site-packages/great_expectations/data_context/data_context/init.py:1 ----> 1 from great_expectations.data_context.data_context.abstract_data_context import ( 2 AbstractDataContext, 3 ) 4 from great_expectations.data_context.data_context.base_data_context import ( 5 BaseDataContext, 6 ) 7 from great_expectations.data_context.data_context.cloud_data_context import ( 8 CloudDataContext, 9 )

File /opt/conda/lib/python3.11/site-packages/great_expectations/data_context/data_context/abstract_data_context.py:71 69 from great_expectations.core.yaml_handler import YAMLHandler 70 from great_expectations.data_asset import DataAsset ---> 71 from great_expectations.data_context.config_validator.yaml_config_validator import ( 72 _YamlConfigValidator, 73 ) 74 from great_expectations.data_context.store import Store, TupleStoreBackend 75 from great_expectations.data_context.store.profiler_store import ProfilerStore

File /opt/conda/lib/python3.11/site-packages/great_expectations/data_context/config_validator/yaml_config_validator.py:20 17 from ruamel.yaml import YAML 19 from great_expectations.alias_types import JSONValues # noqa: TCH001 ---> 20 from great_expectations.checkpoint import Checkpoint, SimpleCheckpoint 21 from great_expectations.core.usage_statistics.anonymizers.anonymizer import Anonymizer 22 from great_expectations.core.usage_statistics.anonymizers.datasource_anonymizer import ( 23 DatasourceAnonymizer, 24 )

File /opt/conda/lib/python3.11/site-packages/great_expectations/checkpoint/init.py:16 1 from ..util import verify_dynamic_loading_support 2 from .actions import ( 3 EmailAction, 4 MicrosoftTeamsNotificationAction, (...) 14 ValidationAction, 15 ) ---> 16 from .checkpoint import Checkpoint, SimpleCheckpoint 17 from .configurator import SimpleCheckpointConfigurator 19 for module_name, package_name in [ 20 (".actions", "great_expectations.checkpoint"), 21 (".checkpoint", "great_expectations.checkpoint"), 22 (".util", "great_expectations.checkpoint"), 23 ]:

File /opt/conda/lib/python3.11/site-packages/great_expectations/checkpoint/checkpoint.py:49 47 from great_expectations.core.config_peer import ConfigOutputModes, ConfigPeer 48 from great_expectations.core.usage_statistics.events import UsageStatsEvents ---> 49 from great_expectations.core.usage_statistics.usage_statistics import ( 50 get_checkpoint_run_usage_statistics, 51 usage_statistics_enabled_method, 52 ) 53 from great_expectations.data_context.cloud_constants import GXCloudRESTResource 54 from great_expectations.data_context.types.base import ( 55 CheckpointConfig, 56 CheckpointValidationConfig, 57 )

File /opt/conda/lib/python3.11/site-packages/great_expectations/core/usage_statistics/usage_statistics.py:29 27 from great_expectations.core.util import nested_update 28 from great_expectations.data_context.types.base import CheckpointConfig ---> 29 from great_expectations.rule_based_profiler.config import RuleBasedProfilerConfig 31 if TYPE_CHECKING: 32 import uuid

File /opt/conda/lib/python3.11/site-packages/great_expectations/rule_based_profiler/init.py:1 ----> 1 from .rule_based_profiler_result import RuleBasedProfilerResult # isort:skip 2 from .rule_based_profiler import BaseRuleBasedProfiler, RuleBasedProfiler # isort:skip

File /opt/conda/lib/python3.11/site-packages/great_expectations/rule_based_profiler/rule_based_profiler_result.py:13 11 from great_expectations.core.domain import Domain # noqa: TCH001 12 from great_expectations.core.usage_statistics.events import UsageStatsEvents ---> 13 from great_expectations.core.usage_statistics.usage_statistics import ( 14 UsageStatisticsHandler, 15 get_expectation_suite_usage_statistics, 16 usage_statistics_enabled_method, 17 ) 18 from great_expectations.core.util import convert_to_json_serializable 19 from great_expectations.rule_based_profiler.helpers.util import ( 20 get_or_create_expectation_suite, 21 )

ImportError: cannot import name 'UsageStatisticsHandler' from partially initialized module 'great_expectations.core.usage_statistics.usage_statistics' (most likely due to a circular import) (/opt/conda/lib/python3.11/site-packages/great_expectations/core/usage_statistics/usage_statistics.py)`

Expected behavior import should succeed

Environment (please complete the following information):

austiezr commented 1 year ago

Hey @ralemy! Thanks for raising this. This should resolve by downgrading to python==3.10, as GX doesn't currently support 3.11.

We are actively investigating the root cause and will address as needed.

dmohns commented 1 year ago

Probably related to https://github.com/great-expectations/great_expectations/issues/5761 ?

I'm encountering the same issue with Python 3.10 and great-expectations==0.15.41. Looking at this and the other issue it seems very hard to reproduce. Can this be related to arm64 architecture?