ImportError BuildConfig from Lightning

romanchemist commented 10 months ago

Bug description

I try to run files from 6.5.4 unit of Lightning tutorial (https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit06-dl-tips/6.5-hparam-opt/6.5-part4-lightning-hpo), but I get the next error: error_screen I use the latest versions of lightning(2.1.2), lightning_hpo(0.0.7). Besides I have successful run of 6.5.3 unit without sweeper. Please tell me how to fix the problem?

What version are you seeing the problem on?

v2.1

How to reproduce the bug

# sweeper.py
import os.path as ops

import optuna
import lightning
#from lightning import LightningApp
#from lightning_hpo import Sweep
import lightning_hpo
from lightning_hpo.algorithm.optuna import OptunaAlgorithm
from lightning_hpo.distributions.distributions import (
    Categorical,
    IntUniform,
    LogUniform,
)

app = LightningApp(
    Sweep(
        script_path=ops.join(ops.dirname(__file__), "./mlp_cli2.py"),
        total_experiments=3,
        parallel_experiments=1,
        distributions={
            "--model.learning_rate": LogUniform(0.001, 0.1),
            "--model.hidden_units": Categorical(["[50, 100]", "[100, 200]"]),
            "--data.batch_size": Categorical([32, 64]),
            "--trainer.max_epochs": IntUniform(1, 3),
        },
        algorithm=OptunaAlgorithm(optuna.create_study(direction="maximize")),
        framework="pytorch_lightning",
    )
)

Error messages and logs

# Error messages and logs here please

ERROR: Found an exception when loading your application from sweeper.py. Please, resolve it to run your app.

Traceback (most recent call last): File "sweeper.py", line 7, in import lightning_hpo File "C:\Users\Roman\anaconda3\envs\AI\lib\site-packages\lightning_hpo__init.py", line 22, in from lightning_hpo.components.sweep import Sweep # noqa: E402 File "C:\Users\Roman\anaconda3\envs\AI\lib\site-packages\lightning_hpo\components\sweep.py", line 6, in from lightning import BuildConfig, CloudCompute, LightningFlow ImportError: cannot import name 'BuildConfig' from 'lightning' (C:\Users\Roman\anaconda3\envs\AI\lib\site-packages\lightning\init__.py)

Environment

Current environment

* CUDA: - GPU: None - available: False - version: None * Lightning: - lightning: 2.1.2 - lightning-cloud: 0.5.57 - lightning-hpo: 0.0.7 - lightning-jupyter: 0.0.2 - lightning-training-studio: 0.0.7 - lightning-utilities: 0.10.0 - pytorch-lightning: 2.1.2 - torch: 2.1.1 - torchmetrics: 1.2.1 - torchvision: 0.16.1+cpu - torchviz: 0.0.2 * Packages: - absl-py: 2.0.0 - aiohttp: 3.9.1 - aiosignal: 1.3.1 - alembic: 1.13.0 - altair: 5.2.0 - annotated-types: 0.6.0 - ansicon: 1.89.0 - antlr4-python3-runtime: 4.9.3 - anyio: 3.7.1 - appdirs: 1.4.4 - argon2-cffi: 21.3.0 - argon2-cffi-bindings: 21.2.0 - arrow: 1.3.0 - asttokens: 2.0.5 - async-lru: 2.0.4 - async-timeout: 4.0.3 - attrs: 23.1.0 - babel: 2.13.1 - backcall: 0.2.0 - backoff: 2.2.1 - backports.functools-lru-cache: 1.6.4 - beautifulsoup4: 4.12.2 - bitsandbytes: 0.41.3 - black: 21.12b0 - bleach: 4.1.0 - blessed: 1.20.0 - blinker: 1.7.0 - boto3: 1.33.10 - botocore: 1.33.10 - brotli: 1.1.0 - cachedmethods: 0.1.4 - cachetools: 5.3.2 - certifi: 2023.7.22 - cffi: 1.15.0 - cgrtools: 4.1.33 - charset-normalizer: 3.3.2 - cimtools: 4.0.12 - click: 8.1.7 - colorama: 0.4.4 - colorlog: 6.8.0 - croniter: 1.3.15 - cycler: 0.10.0 - dataclasses: 0.8 - dateutils: 0.6.12 - deap: 1.3.1 - debugpy: 1.5.1 - decorator: 5.1.1 - deepdiff: 6.7.1 - defusedxml: 0.7.1 - docker-pycreds: 0.4.0 - docstring-parser: 0.15 - entrypoints: 0.3 - exceptiongroup: 1.2.0 - executing: 0.8.2 - fastapi: 0.88.0 - fastjsonschema: 2.19.0 - filelock: 3.3.1 - flask: 3.0.0 - flask-compress: 1.14 - flit-core: 3.6.0 - fqdn: 1.5.1 - frozenlist: 1.4.0 - fsspec: 2023.12.0 - gitdb: 4.0.11 - gitpython: 3.1.40 - google-auth: 2.25.1 - google-auth-oauthlib: 1.1.0 - graphviz: 0.17 - greenlet: 1.1.2 - grpcio: 1.60.0 - h11: 0.14.0 - hiplot: 0.1.34rc200 - httpcore: 1.0.2 - httpx: 0.25.2 - hydra-core: 1.3.2 - idna: 3.6 - importlib-metadata: 4.10.1 - importlib-resources: 5.4.0 - inquirer: 3.1.4 - ipykernel: 6.7.0 - ipython: 8.0.1 - ipython-genutils: 0.2.0 - isoduration: 20.11.0 - itsdangerous: 2.1.2 - jedi: 0.18.1 - jinja2: 3.1.2 - jinxed: 1.2.1 - jmespath: 1.0.1 - joblib: 1.1.0 - json5: 0.9.14 - jsonargparse: 4.27.1 - jsonpointer: 2.4 - jsonschema: 4.20.0 - jsonschema-specifications: 2023.11.2 - jupyter-client: 7.4.9 - jupyter-core: 5.5.0 - jupyter-events: 0.9.0 - jupyter-lsp: 2.2.1 - jupyter-server: 2.12.1 - jupyter-server-terminals: 0.4.4 - jupyterlab: 4.1.0a4 - jupyterlab-pygments: 0.1.2 - jupyterlab-server: 2.25.2 - kiwisolver: 1.3.2 - lightning: 2.1.2 - lightning-cloud: 0.5.57 - lightning-hpo: 0.0.7 - lightning-jupyter: 0.0.2 - lightning-training-studio: 0.0.7 - lightning-utilities: 0.10.0 - llvmlite: 0.37.0 - lxml: 4.6.3 - mako: 1.3.0 - markdown: 3.5.1 - markdown-it-py: 3.0.0 - markupsafe: 2.1.3 - matplotlib: 3.4.3 - matplotlib-inline: 0.1.3 - mdurl: 0.1.2 - mistune: 3.0.2 - mpmath: 1.3.0 - msgpack: 1.0.2 - multidict: 6.0.4 - mypy-extensions: 0.4.3 - nbclient: 0.5.10 - nbconvert: 7.12.0 - nbformat: 5.9.2 - nest-asyncio: 1.5.4 - networkx: 3.2.1 - notebook: 6.4.7 - notebook-shim: 0.2.3 - numpy: 1.22.0 - oauthlib: 3.2.2 - olefile: 0.46 - omegaconf: 2.3.0 - optuna: 3.4.0 - ordered-set: 4.1.0 - overrides: 7.4.0 - packaging: 21.3 - pandas: 1.3.3 - pandocfilters: 1.5.0 - parso: 0.8.3 - pathspec: 0.9.0 - peewee: 3.17.0 - pickleshare: 0.7.5 - pillow: 10.1.0 - pip: 21.3 - platformdirs: 4.1.0 - prometheus-client: 0.12.0 - prompt-toolkit: 3.0.24 - protobuf: 3.20.1 - psutil: 5.9.6 - pure-eval: 0.2.2 - py-mini-racer: 0.6.0 - pyarrow: 14.0.1 - pyasn1: 0.5.1 - pyasn1-modules: 0.3.0 - pycairo: 1.20.1 - pycparser: 2.21 - pydantic: 1.10.13 - pydantic-core: 2.14.5 - pydeck: 0.8.1b0 - pydeprecate: 0.3.2 - pygments: 2.17.2 - pyjnius: 1.4.0 - pyjwt: 2.8.0 - pympler: 1.0.1 - pyparsing: 2.4.7 - pyrsistent: 0.18.1 - python-dateutil: 2.8.2 - python-editor: 1.0.4 - python-json-logger: 2.0.7 - python-multipart: 0.0.6 - pytorch-lightning: 2.1.2 - pytz: 2021.3 - pywin32: 303 - pywinpty: 2.0.12 - pyyaml: 6.0 - pyzmq: 25.1.2 - ray: 1.7.1 - readchar: 4.1.0.dev1 - redis: 3.5.3 - referencing: 0.32.0 - reportlab: 3.5.68 - requests: 2.31.0 - requests-oauthlib: 1.3.1 - rfc3339-validator: 0.1.4 - rfc3986-validator: 0.1.1 - rich: 13.7.0 - rpds-py: 0.13.2 - rsa: 4.9 - s3transfer: 0.8.2 - scikit-learn: 0.24.2 - scipy: 1.7.1 - seaborn: 0.11.2 - semver: 3.0.2 - send2trash: 1.8.2 - sentry-sdk: 1.38.0 - setproctitle: 1.3.3 - setuptools: 58.2.0 - six: 1.16.0 - sklearn-genetic: 0.4.1 - smmap: 5.0.1 - sniffio: 1.3.0 - soupsieve: 2.5 - sqlalchemy: 2.0.23 - sqlite-web: 0.6.1 - sqlmodel: 0.0.14 - stack-data: 0.1.4 - starlette: 0.22.0 - starsessions: 1.3.0 - streamlit: 1.12.0 - sympy: 1.12 - tensorboard: 2.15.1 - tensorboard-data-server: 0.7.2 - tensorboardx: 2.6.2.2 - terminado: 0.12.1 - testpath: 0.5.0 - threadpoolctl: 3.0.0 - tinycss2: 1.2.1 - toml: 0.10.2 - tomli: 1.2.2 - toolz: 0.12.0 - torch: 2.1.1 - torchmetrics: 1.2.1 - torchvision: 0.16.1+cpu - torchviz: 0.0.2 - tornado: 6.4 - tqdm: 4.62.3 - traitlets: 5.14.0 - typed-ast: 1.5.1 - types-python-dateutil: 2.8.19.14 - typeshed-client: 2.4.0 - typing-extensions: 4.9.0rc1 - tzdata: 2023.3 - tzlocal: 5.2 - uri-template: 1.3.0 - urllib3: 1.26.18 - uvicorn: 0.24.0.post1 - validators: 0.22.0 - wandb: 0.16.1 - watchdog: 3.0.0 - watermark: 2.4.3 - wcwidth: 0.2.5 - webcolors: 1.13 - webencodings: 0.5.1 - websocket-client: 1.7.0 - websockets: 11.0.3 - werkzeug: 3.0.1 - wheel: 0.37.0 - yarl: 1.9.3 - zipp: 3.7.0 * System: - OS: Windows - architecture: - 64bit - WindowsPE - processor: Intel64 Family 6 Model 142 Stepping 11, GenuineIntel - python: 3.9.7 - release: 10 - version: 10.0.22621

More info

No response

awaelchli commented 10 months ago

@romanchemist In the latest version of lightning, we made app an extra. So you'll have to install via

pip install -U lightning[data]

Also note that lightning-hpo is unmaintained, at the moment there are no updates planned for this package.

romanchemist commented 10 months ago

Thanks! I've done it. It was installed some packages(fastapi-0.104.1 lightning-cloud-0.5.52 starlette-0.27.0 torchdata-0.7.1). But I have the same error..

awaelchli commented 10 months ago

@romanchemist I think you can only use lightning-hpo with lightning 1.8.6: https://github.com/Lightning-Universe/Training-Studio_app/blob/master/requirements.txt So you'd have to downgrade the version. This lightning-hpo repo is archived and no longer being developed, so I'm not sure how successful you will be though since it's quite old.

Closing the issue here, since we can't do much about it here on the pytorch-lightning side, sorry :(

Lightning-AI / pytorch-lightning