search

sspaeti-com / practical-data-engineering

Practical Data Engineering: A Hands-On Real-Estate Project Guide
https://ssp.sh/blog/data-engineering-project-in-twenty-minutes
475 stars 80 forks source link

Code location import error #3

Open maxestorr opened 1 month ago

maxestorr commented 1 month ago

The following block of code is causing the code location to error for some reason:

File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/realestate/pipelines.py", line 93, in 

    def collect_properties(properties: List[PropertyDataFrame]) -> List[PropertyDataFrame]:  # type: ignore

Error message:

dagster._core.errors.DagsterUserCodeLoadError: Error occurred during the loading of Dagster definitions in
executable_path=/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/bin/python3, module_name=realestate, working_directory=/home/max/dev/practical-data-engineering/src/pipelines/real-estate

  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_grpc/server.py", line 408, in __init__
    self._loaded_repositories: Optional[LoadedRepositories] = LoadedRepositories(
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_grpc/server.py", line 235, in __init__
    with user_code_error_boundary(
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_core/errors.py", line 297, in user_code_error_boundary
    raise new_error from e

The above exception was caused by the following exception:
TypeError: Parameters to generic types must be types. Got <dagster._core.types.dagster_type.DagsterType object at 0x7f710117c820>.

  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_core/errors.py", line 287, in user_code_error_boundary
    yield
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_grpc/server.py", line 242, in __init__
    loadable_targets = get_loadable_targets(
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_grpc/utils.py", line 50, in get_loadable_targets
    else loadable_targets_from_python_module(module_name, working_directory)
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_core/workspace/autodiscovery.py", line 35, in loadable_targets_from_python_module
    module = load_python_module(
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/.venv/lib/python3.10/site-packages/dagster/_core/code_pointer.py", line 134, in load_python_module
    return importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/realestate/__init__.py", line 8, in <module>
    from .pipelines import scrape_realestate, resource_def
  File "/home/max/dev/practical-data-engineering/src/pipelines/real-estate/realestate/pipelines.py", line 93, in <module>
    def collect_properties(properties: List[PropertyDataFrame]) -> List[PropertyDataFrame]:  # type: ignore
  File "/usr/lib/python3.10/typing.py", line 312, in inner
    return func(*args, **kwds)
  File "/usr/lib/python3.10/typing.py", line 1143, in __getitem__
    params = tuple(_type_check(p, msg) for p in params)
  File "/usr/lib/python3.10/typing.py", line 1143, in <genexpr>
    params = tuple(_type_check(p, msg) for p in params)
  File "/usr/lib/python3.10/typing.py", line 176, in _type_check
    raise TypeError(f"{msg} Got {arg!r:.100}.")
maxestorr commented 1 month ago

Really bizarre seems like it thinks dagster._core.types.dagster_type.DagsterType isn't a type?

maxestorr commented 1 month ago

System info btw I'm trying to run this on WSL2 running OS: Ubuntu 22.04.4 LTS on Windows 10 x86_64, Kernel: 5.15.146.1-microsoft-standard-WSL2, this happened right after cloning, creating a venv, pip installing dependencies, running minio, and finally running dagster dev.