Netflix / metaflow

Open Source Platform for developing, scaling and deploying serious ML, AI, and data science systems
https://metaflow.org
Apache License 2.0
8.13k stars 765 forks source link

Fail to `import metaflow` on Windows #2110

Open asaff1 opened 2 days ago

asaff1 commented 2 days ago

Environment: metaflow version: 2.12.25 Windows Python 3.9

Description: This line results in NameError: check_output(["where", "/Q", "git"]) https://github.com/Netflix/metaflow/blob/5baaf3ce07b65142c6ea41125575f37ad32b4e41/metaflow/metaflow_version.py#L30C13-L30C25 it should do instead: subprocess.check_output(["where", "/Q", "git"])

Fixed that, I'm now seeing this issue:

Traceback (most recent call last):
  File "C:\Python39\lib\site-packages\metaflow\extension_support\plugins.py", line 137, in resolve_plugins
    plugin_module = importlib.import_module(path)
  File "C:\Python39\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\Python39\lib\site-packages\metaflow\plugins\aws\batch\batch_decorator.py", line 23, in <module>
    from metaflow.sidecar import Sidecar
  File "C:\Python39\lib\site-packages\metaflow\sidecar\__init__.py", line 1, in <module>
    from .sidecar import Sidecar
  File "C:\Python39\lib\site-packages\metaflow\sidecar\sidecar.py", line 1, in <module>
    from .sidecar_subprocess import SidecarSubProcess
  File "C:\Python39\lib\site-packages\metaflow\sidecar\sidecar_subprocess.py", line 4, in <module>
    import fcntl
ModuleNotFoundError: No module named 'fcntl'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Python39\Scripts\metaflow.exe\__main__.py", line 4, in <module>
  File "C:\Python39\lib\site-packages\metaflow\__init__.py", line 113, in <module>
    from .plugins.datatools import S3
  File "C:\Python39\lib\site-packages\metaflow\plugins\__init__.py", line 159, in <module>
    STEP_DECORATORS = resolve_plugins("step_decorator")
  File "C:\Python39\lib\site-packages\metaflow\extension_support\plugins.py", line 139, in resolve_plugins
    raise ValueError(
ValueError: Cannot locate step_decorator plugin 'batch' at 'metaflow.plugins.aws.batch.batch_decorator

Is the library supports Windows?

romain-intel commented 22 hours ago

Metaflow is not supported on Windows; it is supported on Mac and Linux. I believe it may work with WSL though.