dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.51k stars 1.45k forks source link

AttributeError: 'InputMappingNode' object has no attribute 'shape' #16030

Open moghadas76 opened 1 year ago

moghadas76 commented 1 year ago

Dagster version

1.4.5

What's the issue?

Traceback (most recent call last): File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_grpc/server.py", line 290, in __init__ self._loaded_repositories: Optional[LoadedRepositories] = LoadedRepositories( File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_grpc/server.py", line 134, in __init__ loadable_targets = get_loadable_targets( File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_grpc/utils.py", line 47, in get_loadable_targets else loadable_targets_from_python_module(module_name, working_directory) File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_core/workspace/autodiscovery.py", line 35, in loadable_targets_from_python_module module = load_python_module( File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_core/code_pointer.py", line 135, in load_python_module return importlib.import_module(module_name) File "/home/seyed/miniconda3/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 688, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/home/seyed/PycharmProjects/step/taffic-data-pipeline/taffic_data_pipeline/__init__.py", line 4, in <module> from .transformations import get_time_series_data_and_adj File "/home/seyed/PycharmProjects/step/taffic-data-pipeline/taffic_data_pipeline/transformations.py", line 54, in <module> def get_x_y(pems_bay: pd.DataFrame): File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_core/definitions/decorators/asset_decorator.py", line 888, in graph_asset op_graph = graph( File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_core/definitions/decorators/graph_decorator.py", line 79, in __call__ ) = do_composition( File "/home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_core/definitions/composition.py", line 1035, in do_composition output = fn(**kwargs) File "/home/seyed/PycharmProjects/step/taffic-data-pipeline/taffic_data_pipeline/transformations.py", line 68, in get_x_y x,y = generate_graph_seq2seq_io_data(pems_bay, x_offsets=x_offsets, y_offsets=y_offsets,add_day_in_week=False, add_time_in_day=True) File "/home/seyed/PycharmProjects/step/taffic-data-pipeline/taffic_data_pipeline/transformations.py", line 25, in generate_graph_seq2seq_io_data num_samples, num_nodes = df.shape AttributeError: 'InputMappingNode' object has no attribute 'shape' 2023-08-23 11:21:47 +0200 - dagster.code_server - INFO - Started Dagster code server for module taffic_data_pipeline in process 97848 /home/seyed/miniconda3/lib/python3.10/site-packages/dagster/_core/workspace/context.py:610: UserWarning: Error loading repository location taffic_data_pipeline:AttributeError: 'InputMappingNode' object has no attribute 'shape'

@graph_asset def get_x_y(pems_bay: pd.DataFrame): """ x,y = generate_graph_seq2seq_io_data(pems_bay, x_offsets=x_offsets, y_offsets=y_offsets,add_day_in_week=False, add_time_in_day=True) # raise happens

What did you expect to happen?

Preserve the asset (pems_bay) attributes as DataFrame Object

How to reproduce?

No response

Deployment type

None

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

jamiedemaria commented 1 year ago

hi @moghadas76 can you share the code for generate_graph_seq2seq_io_data?

Also if you use three ticks ``` you'll be able to do code block formatting, which will make everything easier to read

Example:

@asset 
def my_cool_asset():
    return 1