search

Lightning-AI / pytorch-lightning

Pretrain, finetune ANY AI model of ANY size on multiple GPUs, TPUs with zero code changes.
https://lightning.ai
Apache License 2.0
28.38k stars 3.38k forks source link

Unable to see build logs using the CLI #15139

Closed krishnakalyan3 closed 1 year ago

krishnakalyan3 commented 2 years ago

Bug description

lightning show logs app_name shows only the last couple of lines.

Slack thread: https://pytorch-lightning.slack.com/archives/CRBLFHY79/p1665487921218359

How to reproduce the bug

git clone https://github.com/Lightning-AI/lightning-hpo && cd lightning-hpo

pip install -e . -r requirements.txt --find-links https://download.pytorch.org/whl/cpu/torch_stable.html
python -m lightning run app app.py

lightning show logs app_name

Error messages and logs

Only the last 2 lines are printed

lightning show logs research_studio_app                                                                                              127 err  demo-hpo py 
flow 10/11/2022 10:08:38 INFO: Your app has started. View it in your browser: http://0.0.0.0:8080/view
flow 10/11/2022 10:08:44 INFO: The Training Studio App is ready !

Environment

* CUDA:
    - GPU:               None
    - available:         False
    - version:           None
* Lightning:
    - lightning:         2022.10.12
    - lightning-cloud:   0.5.10
    - lightning-hpo:     0.0.2
    - lightning-jupyter: 0.0.2
    - lightning-transformers: 0.2.3
    - lightning-utilities: 0.3.0
    - pytorch-lightning: 1.7.7
    - torch:             1.12.1
    - torchmetrics:      0.10.0
    - torchvision:       0.13.1
* Packages:
    - absl-py:           1.2.0
    - aiobotocore:       2.4.0
    - aiohttp:           3.8.3
    - aioitertools:      0.11.0
    - aiosignal:         1.2.0
    - alembic:           1.8.1
    - altair:            4.2.0
    - anyio:             3.6.1
    - appnope:           0.1.3
    - argon2-cffi:       21.3.0
    - argon2-cffi-bindings: 21.2.0
    - arrow:             1.2.3
    - asttokens:         2.0.8
    - async-timeout:     4.0.2
    - attrs:             22.1.0
    - autopage:          0.5.1
    - babel:             2.10.3
    - backcall:          0.2.0
    - backports.zoneinfo: 0.2.1
    - beautifulsoup4:    4.11.1
    - bleach:            5.0.1
    - blinker:           1.5
    - botocore:          1.27.59
    - brotli:            1.0.9
    - cachetools:        5.2.0
    - certifi:           2022.9.24
    - cffi:              1.15.1
    - charset-normalizer: 2.1.1
    - click:             8.1.3
    - cliff:             4.0.0
    - cmaes:             0.8.2
    - cmd2:              2.4.2
    - colorlog:          6.7.0
    - commonmark:        0.9.1
    - croniter:          1.3.7
    - datasets:          2.5.2
    - debugpy:           1.6.3
    - decorator:         5.1.1
    - deepdiff:          5.8.1
    - defusedxml:        0.7.1
    - dill:              0.3.5.1
    - dnspython:         2.2.1
    - docker-pycreds:    0.4.0
    - docstring-parser:  0.15
    - email-validator:   1.3.0
    - entrypoints:       0.4
    - executing:         1.1.1
    - fastapi:           0.85.0
    - fastjsonschema:    2.16.2
    - filelock:          3.8.0
    - fire:              0.4.0
    - flask:             2.2.2
    - flask-compress:    1.13
    - frozenlist:        1.3.1
    - fsspec:            2022.8.2
    - gitdb:             4.0.9
    - gitpython:         3.1.29
    - google-auth:       2.12.0
    - google-auth-oauthlib: 0.4.6
    - grpcio:            1.49.1
    - h11:               0.14.0
    - hiplot:            0.1.33
    - httptools:         0.5.0
    - huggingface-hub:   0.10.1
    - idna:              3.4
    - importlib-metadata: 4.13.0
    - importlib-resources: 5.10.0
    - ipykernel:         6.16.0
    - ipython:           8.5.0
    - ipython-genutils:  0.2.0
    - itsdangerous:      2.1.2
    - jedi:              0.18.1
    - jinja2:            3.1.2
    - jmespath:          1.0.1
    - joblib:            1.2.0
    - json5:             0.9.10
    - jsonargparse:      4.15.1
    - jsonschema:        4.16.0
    - jupyter-client:    7.3.5
    - jupyter-core:      4.11.1
    - jupyter-server:    1.19.1
    - jupyterlab:        3.4.8
    - jupyterlab-pygments: 0.2.2
    - jupyterlab-server: 2.15.2
    - lightning:         2022.10.12
    - lightning-cloud:   0.5.10
    - lightning-hpo:     0.0.2
    - lightning-jupyter: 0.0.2
    - lightning-transformers: 0.2.3
    - lightning-utilities: 0.3.0
    - mako:              1.2.3
    - markdown:          3.4.1
    - markupsafe:        2.1.1
    - matplotlib-inline: 0.1.6
    - mistune:           2.0.4
    - multidict:         6.0.2
    - multiprocess:      0.70.13
    - nbclassic:         0.4.5
    - nbclient:          0.7.0
    - nbconvert:         7.2.1
    - nbformat:          5.7.0
    - nest-asyncio:      1.5.6
    - nltk:              3.7
    - notebook:          6.4.12
    - notebook-shim:     0.1.0
    - numpy:             1.23.3
    - oauthlib:          3.2.1
    - optuna:            3.0.3
    - ordered-set:       4.1.0
    - orjson:            3.8.0
    - packaging:         21.3
    - pandas:            1.5.0
    - pandocfilters:     1.5.0
    - parso:             0.8.3
    - pathtools:         0.1.2
    - pbr:               5.10.0
    - peewee:            3.15.3
    - pexpect:           4.8.0
    - pickleshare:       0.7.5
    - pillow:            9.2.0
    - pip:               22.2.2
    - pkgutil-resolve-name: 1.3.10
    - prettytable:       3.4.1
    - prometheus-client: 0.14.1
    - promise:           2.3
    - prompt-toolkit:    3.0.31
    - protobuf:          3.19.6
    - psutil:            5.9.2
    - ptyprocess:        0.7.0
    - pure-eval:         0.2.2
    - pyarrow:           9.0.0
    - pyasn1:            0.4.8
    - pyasn1-modules:    0.2.8
    - pycparser:         2.21
    - pydantic:          1.10.2
    - pydeck:            0.8.0b3
    - pydeprecate:       0.3.2
    - pygments:          2.13.0
    - pyjwt:             2.5.0
    - pympler:           1.0.1
    - pyparsing:         3.0.9
    - pyperclip:         1.8.2
    - pyrsistent:        0.18.1
    - python-dateutil:   2.8.2
    - python-dotenv:     0.21.0
    - python-multipart:  0.0.5
    - pytorch-lightning: 1.7.7
    - pytz:              2022.4
    - pytz-deprecation-shim: 0.1.0.post0
    - pyyaml:            6.0
    - pyzmq:             24.0.1
    - regex:             2022.9.13
    - requests:          2.28.1
    - requests-oauthlib: 1.3.1
    - responses:         0.18.0
    - rich:              12.6.0
    - rsa:               4.9
    - s3fs:              2022.8.2
    - scikit-learn:      1.1.2
    - scipy:             1.8.1
    - semver:            2.13.0
    - send2trash:        1.8.0
    - sentencepiece:     0.1.97
    - sentry-sdk:        1.9.10
    - setproctitle:      1.3.2
    - setuptools:        65.4.1
    - shortuuid:         1.0.9
    - six:               1.16.0
    - sklearn:           0.0
    - smmap:             5.0.0
    - sniffio:           1.3.0
    - soupsieve:         2.3.2.post1
    - sqlalchemy:        1.4.41
    - sqlalchemy2-stubs: 0.0.2a28
    - sqlite-web:        0.4.0
    - sqlmodel:          0.0.8
    - stack-data:        0.5.1
    - starlette:         0.20.4
    - starsessions:      1.3.0
    - stevedore:         4.0.0
    - streamlit:         1.13.0
    - tensorboard:       2.10.1
    - tensorboard-data-server: 0.6.1
    - tensorboard-plugin-wit: 1.8.1
    - termcolor:         2.0.1
    - terminado:         0.16.0
    - threadpoolctl:     3.1.0
    - tinycss2:          1.1.1
    - tokenizers:        0.13.1
    - toml:              0.10.2
    - tomli:             2.0.1
    - toolz:             0.12.0
    - torch:             1.12.1
    - torchmetrics:      0.10.0
    - torchvision:       0.13.1
    - tornado:           6.2
    - tqdm:              4.64.1
    - traitlets:         5.4.0
    - transformers:      4.23.1
    - typing-extensions: 4.4.0
    - tzdata:            2022.4
    - tzlocal:           4.2
    - ujson:             5.5.0
    - urllib3:           1.26.12
    - uvicorn:           0.18.3
    - uvloop:            0.17.0
    - validators:        0.20.0
    - wandb:             0.13.4
    - watchfiles:        0.17.0
    - wcwidth:           0.2.5
    - webencodings:      0.5.1
    - websocket-client:  1.4.1
    - websockets:        10.3
    - werkzeug:          2.2.2
    - wheel:             0.37.1
    - wrapt:             1.14.1
    - xxhash:            3.0.0
    - yarl:              1.8.1
    - zipp:              3.9.0
* System:
    - OS:                Darwin
    - architecture:
        - 64bit
        -
    - processor:         arm
    - python:            3.8.13
    - version:           Darwin Kernel Version 21.6.0: Sat Jun 18 17:07:22 PDT 2022; root:xnu-8020.140.41~1/RELEASE_ARM64_T6000```

### More info

_No response_

cc: @hhsecond

Borda commented 1 year ago

@tchaton could you pls check if this was fixed already?

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions - the Lightning Team!