iterative / dvc

🦉 Data Versioning and ML Experiments
https://dvc.org
Apache License 2.0
13.89k stars 1.18k forks source link

json output is not valid, due to colorized ANSI escape codes #10430

Closed aaltemara closed 5 months ago

aaltemara commented 6 months ago

Bug Report

Issue name

dvc status --json: not parsable due to color escape codes

Description

Expectation is that --json produces json-compatible output. This is not the case, due to ANSI escape codes that lead to invalid json.

Reproduce

  1. dvc init
  2. dvc data status --json -q > ~/out
  3. vim ~/out (Shows) ^[[1m{^[[0m^[[1m}^[[0m ^[[0m

Expected

(No escape codes) {}

I expect to be able to pipe the output of a --json command into 'jq' without json formatting errors

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 3.50.2 (deb)
-------------------------
Platform: Python 3.10.8 on Linux-4.18.0-425.10.1.el8_7.x86_64-x86_64-with-glibc2.31
Subprojects:

Supports:
        azure (adlfs = 2024.4.1, knack = 0.11.0, azure-identity = 1.16.0),
        gdrive (pydrive2 = 1.19.0),
        gs (gcsfs = 2024.3.1),
        hdfs (fsspec = 2024.3.1, pyarrow = 16.0.0),
        http (aiohttp = 3.9.5, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.9.5, aiohttp-retry = 2.8.3),
        oss (ossfs = 2023.12.0),
        s3 (s3fs = 2024.3.1, boto3 = 1.34.69),
        ssh (sshfs = 2024.4.1),
        webdav (webdav4 = 0.9.8),
        webdavs (webdav4 = 0.9.8),
        webhdfs (fsspec = 2024.3.1)
Config:
        Global: /root/.config/dvc
        System: /etc/xdg/dvc
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: None
Workspace directory: xfs on /dev/mapper/vg01-lv01
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/113fba86254417b4ef5da622c5281b19

Additional Information (if any):

shcheklein commented 5 months ago

I can't reproduce this right away, I think we have some logic that disables colorize:

def setup(level: int = logging.INFO, log_colors: bool = True) -> None:
    colorama.init()

    color_out = log_colors and bool(sys.stdout) and sys.stdout.isatty()
    color_err = log_colors and bool(sys.stderr) and sys.stderr.isatty()
...

from dvc/logger.py

Moreover, -q doesn't produce (expected any results for me).

Is there anything specific about your setup? What terminal are you using?