fugue-project / fugue

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
https://fugue-tutorials.readthedocs.io/
Apache License 2.0
1.92k stars 94 forks source link

[QUESTION] Ray Example from Docs does not work #512

Closed juanitorduz closed 10 months ago

juanitorduz commented 10 months ago

Hi! I am trying to reproduce the RAY example from https://fugue-tutorials.readthedocs.io/tutorials/beginner/execution_engine.html, and I get an error:

image

ValueError: ray

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
File [~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/fugue/execution/factory.py:393](https://file+.vscode-resource.vscode-cdn.net/Users/juanitorduz/Documents/wm-forecasting-research/notebooks/research/~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/fugue/execution/factory.py:393), in parse_execution_engine(engine, conf, **kwargs)
    392 try:
--> 393     return to_instance(engine, ExecutionEngine, kwargs=dict(conf=conf, **kwargs))
    394 except Exception as e:

File [~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/triad/utils/convert.py:290](https://file+.vscode-resource.vscode-cdn.net/Users/juanitorduz/Documents/wm-forecasting-research/notebooks/research/~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/triad/utils/convert.py:290), in to_instance(s, expected_base_type, args, kwargs, global_vars, local_vars)
    289 if isinstance(s, (str, type)):
--> 290     t = to_type(s, expected_base_type, global_vars, local_vars)
    291     return t(*args, **kwargs)

File [~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/triad/utils/convert.py:250](https://file+.vscode-resource.vscode-cdn.net/Users/juanitorduz/Documents/wm-forecasting-research/notebooks/research/~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/triad/utils/convert.py:250), in to_type(s, expected_base_type, global_vars, local_vars)
    249 if isinstance(s, str):
--> 250     return str_to_type(s, expected_base_type, global_vars, local_vars)
    251 if isinstance(s, type):

File [~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/triad/utils/convert.py:190](https://file+.vscode-resource.vscode-cdn.net/Users/juanitorduz/Documents/wm-forecasting-research/notebooks/research/~/.local/share/virtualenvs/wm-forecasting-research-YZWP1C_I/lib/python3.11/site-packages/triad/utils/convert.py:190), in str_to_type(s, expected_base_type, global_vars, local_vars)
    189 except ValueError:
...
    396         f"Fugue execution engine is not recognized ({engine}, {conf}, {kwargs})."
    397         " You may need to register a parser for it."
    398     ) from e

FuguePluginsRegistrationError: Fugue execution engine is not recognized (ray, None, {}). You may need to register a parser for it.

It does work with engine="spark" in the transform function. What am I missing? Thanks!

juanitorduz commented 10 months ago

OMG I was missing the extras from the README (I suggest adding them more visible on the tutorial pages) 🤦 .

Sorry 🤗

kvnkho commented 10 months ago

Glad you figured it out! Let us know if you need any more help :)