search

kislyuk / yq

Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents
https://kislyuk.github.io/yq/
Apache License 2.0
2.53k stars 81 forks source link

test output too noisy #161

Closed 0-wiz-0 closed 1 year ago

0-wiz-0 commented 1 year ago

I've packaged yq 3.1.1 and wanted to run the self tests to check if it's working fine. In the Makefile I found a test target. Running it gives me a lot of output, and it's not clear to me that the tests are working fine - in particular there are a lot of jq usage lines.

Is the program working fine or not?

Can you please make the test output clearer?

Here's the test log using Python 3.11.2 and jq-1.6.

test_abbrev_opt_collisions (__main__.TestYq.test_abbrev_opt_collisions) ... ok
test_datetimes (__main__.TestYq.test_datetimes) ... [
  "2016-12-20T22:07:36Z"
]
"2016-12-20"
ok
test_entity_expansion_defense (__main__.TestYq.test_entity_expansion_defense) ... /usr/pkg/lib/python3.11/subprocess.py:1125: ResourceWarning: subprocess 21014 is still running
  _warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/scratch/textproc/yq/work/yq-3.1.1/test/test.py:62: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='utf-8'>
  self.assertIn(e.code, expect_exit_codes)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ok
test_explicit_doc_markers (__main__.TestYq.test_explicit_doc_markers) ... ok
test_in_place (__main__.TestYq.test_in_place) ... /scratch/textproc/yq/work/yq-3.1.1/yq/__init__.py:148: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpanxjof5d' mode='w' encoding='utf-8'>
  yq(input_streams=[input_stream], output_stream=DeferredOutputStream(input_stream.name), **yq_args)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/scratch/textproc/yq/work/yq-3.1.1/yq/__init__.py:148: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmp_g02xelw' mode='w' encoding='utf-8'>
  yq(input_streams=[input_stream], output_stream=DeferredOutputStream(input_stream.name), **yq_args)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
jq: error: syntax error, unexpected '/', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
/tmp/tmpanxjof5d
jq: 1 compile error
ok
test_jq16_arg_passthrough (__main__.TestYq.test_jq16_arg_passthrough) ... {}
ok
test_multidocs (__main__.TestYq.test_multidocs) ... ok
test_roundtrip_yaml (__main__.TestYq.test_roundtrip_yaml) ... ok
test_short_option_separation (__main__.TestYq.test_short_option_separation) ... {
  "a": 1
}
{"a":1}
ok
test_tomlq (__main__.TestYq.test_tomlq) ... {
  "foo": {
    "bar": 1
  }
}
{
  "foo": {
    "bar": "2020-02-20"
  }
}
ok
test_unrecognized_tags (__main__.TestYq.test_unrecognized_tags) ... "bar"
ok
test_xq (__main__.TestYq.test_xq) ... {
  "foo": null
}
ok
test_xq_dtd (__main__.TestYq.test_xq_dtd) ... ok
test_yaml_1_1_octals (__main__.TestYq.test_yaml_1_1_octals) ... ok
test_yaml_1_2 (__main__.TestYq.test_yaml_1_2) ... "11:12:13"
ok
test_yaml_1_2_octals (__main__.TestYq.test_yaml_1_2_octals)
YAML 1.2 octals not yet implemented ... expected failure
test_yaml_merge (__main__.TestYq.test_yaml_merge) ... ok
test_yaml_type_tags (__main__.TestYq.test_yaml_type_tags) ... {
  "example": "Zm9vYmFyCg=="
}
{
  "example": {
    "Boston Red Sox": null,
    "Detroit Tigers": null,
    "New York Yankees": null
  }
}
ok
test_yq (__main__.TestYq.test_yq) ... usage: yq [options] <jq filter> [input file...]
yq: Command-line YAML processor - jq wrapper for YAML documents

yq transcodes YAML documents to JSON and passes them to jq.
See https://github.com/kislyuk/yq for more information.

positional arguments:
  jq_filter
  files

options:
  -h, --help            show this help message and exit
  --yaml-output, --yml-output, -y
                        Transcode jq JSON output back into YAML and emit it
  --yaml-roundtrip, --yml-roundtrip, -Y
                        Transcode jq JSON output back into YAML and emit it. Preserve YAML tags and styles by representing them as extra items in their enclosing mappings and sequences while in JSON. This option is incompatible with jq filters that do not expect these extra items.
  --yaml-output-grammar-version {1.1,1.2}, --yml-out-ver {1.1,1.2}
                        When using --yaml-output, specify output grammar (the default is 1.1 and will be changed to 1.2 in a future version). Setting this to 1.2 will cause strings like 'on' and 'no' to be emitted unquoted.
  --width WIDTH, -w WIDTH
                        When using --yaml-output, specify string wrap width
  --indentless-lists, --indentless
                        When using --yaml-output, indent block style lists (sequences) with 0 spaces instead of 2
  --in-place, -i        Edit files in place (no backup - use caution)
  --version             show program's version number and exit

jq - commandline JSON processor [version 1.6]

Usage:  jq [options] <jq filter> [file...]
        jq [options] --args <jq filter> [strings...]
        jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

        $ echo '{"foo": 0}' | jq .
        {
                "foo": 0
        }

Some of the options include:
  -c               compact instead of pretty-printed output;
  -n               use `null` as the single input value;
  -e               set the exit status code based on the output;
  -s               read (slurp) all inputs into an array; apply filter to it;
  -r               output raw strings, not JSON texts;
  -R               read raw strings, not JSON texts;
  -C               colorize JSON;
  -M               monochrome (don't colorize JSON);
  -S               sort keys of objects on output;
  --tab            use tabs for indentation;
  --arg a v        set variable $a to value <v>;
  --argjson a v    set variable $a to JSON value <v>;
  --slurpfile a f  set variable $a to an array of JSON texts read from <f>;
  --rawfile a f    set variable $a to a string consisting of the contents of <f>;
  --args           remaining arguments are string arguments, not files;
  --jsonargs       remaining arguments are JSON arguments, not files;
  --               terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.
usage: yq [options] <jq filter> [input file...]
yq: Command-line YAML processor - jq wrapper for YAML documents

yq transcodes YAML documents to JSON and passes them to jq.
See https://github.com/kislyuk/yq for more information.

positional arguments:
  jq_filter
  files

options:
  -h, --help            show this help message and exit
  --yaml-output, --yml-output, -y
                        Transcode jq JSON output back into YAML and emit it
  --yaml-roundtrip, --yml-roundtrip, -Y
                        Transcode jq JSON output back into YAML and emit it. Preserve YAML tags and styles by representing them as extra items in their enclosing mappings and sequences while in JSON. This option is incompatible with jq filters that do not expect these extra items.
  --yaml-output-grammar-version {1.1,1.2}, --yml-out-ver {1.1,1.2}
                        When using --yaml-output, specify output grammar (the default is 1.1 and will be changed to 1.2 in a future version). Setting this to 1.2 will cause strings like 'on' and 'no' to be emitted unquoted.
  --width WIDTH, -w WIDTH
                        When using --yaml-output, specify string wrap width
  --indentless-lists, --indentless
                        When using --yaml-output, indent block style lists (sequences) with 0 spaces instead of 2
  --in-place, -i        Edit files in place (no backup - use caution)
  --version             show program's version number and exit

jq - commandline JSON processor [version 1.6]

Usage:  jq [options] <jq filter> [file...]
        jq [options] --args <jq filter> [strings...]
        jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

        $ echo '{"foo": 0}' | jq .
        {
                "foo": 0
        }

Some of the options include:
  -c               compact instead of pretty-printed output;
  -n               use `null` as the single input value;
  -e               set the exit status code based on the output;
  -s               read (slurp) all inputs into an array; apply filter to it;
  -r               output raw strings, not JSON texts;
  -R               read raw strings, not JSON texts;
  -C               colorize JSON;
  -M               monochrome (don't colorize JSON);
  -S               sort keys of objects on output;
  --tab            use tabs for indentation;
  --arg a v        set variable $a to value <v>;
  --argjson a v    set variable $a to JSON value <v>;
  --slurpfile a f  set variable $a to an array of JSON texts read from <f>;
  --rawfile a f    set variable $a to a string consisting of the contents of <f>;
  --args           remaining arguments are string arguments, not files;
  --jsonargs       remaining arguments are JSON arguments, not files;
  --               terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.
{}
3
1
ok
test_yq_arg_handling (__main__.TestYq.test_yq_arg_handling) ... {}
{
  "yaml_struct": [
    {
      "key_1": "$.*?/|\\^(){}+@[]&_-",
      "key_2": [
        "$.*?/|\\^(){}+@[]&_-",
        "{ }",
        "value23"
      ],
      "key_3": "value31",
      "key_4": [
        "value41",
        "value42"
      ],
      "key_5": "value51"
    },
    {
      "key_1_": "value12",
      "_key_2": [
        "value24",
        "value25"
      ],
      "_key__3": "$.*?/|\\^(){}+@[]&_-"
    }
  ]
}
{
  "yaml_struct": [
    {
      "key_1": "$.*?/|\\^(){}+@[]&_-",
      "key_2": [
        "$.*?/|\\^(){}+@[]&_-",
        "{ }",
        "value23"
      ],
      "key_3": "value31",
      "key_4": [
        "value41",
        "value42"
      ],
      "key_5": "value51"
    },
    {
      "key_1_": "value12",
      "_key_2": [
        "value24",
        "value25"
      ],
      "_key__3": "$.*?/|\\^(){}+@[]&_-"
    }
  ]
}
{
  "yaml_struct": [
    {
      "key_1": "$.*?/|\\^(){}+@[]&_-",
      "key_2": [
        "$.*?/|\\^(){}+@[]&_-",
        "{ }",
        "value23"
      ],
      "key_3": "value31",
      "key_4": [
        "value41",
        "value42"
      ],
      "key_5": "value51"
    },
    {
      "key_1_": "value12",
      "_key_2": [
        "value24",
        "value25"
      ],
      "_key__3": "$.*?/|\\^(){}+@[]&_-"
    }
  ]
}
{}
{
  "yaml_struct": [
    {
      "key_1": "$.*?/|\\^(){}+@[]&_-",
      "key_2": [
        "$.*?/|\\^(){}+@[]&_-",
        "{ }",
        "value23"
      ],
      "key_3": "value31",
      "key_4": [
        "value41",
        "value42"
      ],
      "key_5": "value51"
    },
    {
      "key_1_": "value12",
      "_key_2": [
        "value24",
        "value25"
      ],
      "_key__3": "$.*?/|\\^(){}+@[]&_-"
    }
  ]
}
{
  "yaml_struct": [
    {
      "key_1": "$.*?/|\\^(){}+@[]&_-",
      "key_2": [
        "$.*?/|\\^(){}+@[]&_-",
        "{ }",
        "value23"
      ],
      "key_3": "value31",
      "key_4": [
        "value41",
        "value42"
      ],
      "key_5": "value51"
    },
    {
      "key_1_": "value12",
      "_key_2": [
        "value24",
        "value25"
      ],
      "_key__3": "$.*?/|\\^(){}+@[]&_-"
    }
  ]
}
{
  "yaml_struct": [
    {
      "key_1": "$.*?/|\\^(){}+@[]&_-",
      "key_2": [
        "$.*?/|\\^(){}+@[]&_-",
        "{ }",
        "value23"
      ],
      "key_3": "value31",
      "key_4": [
        "value41",
        "value42"
      ],
      "key_5": "value51"
    },
    {
      "key_1_": "value12",
      "_key_2": [
        "value24",
        "value25"
      ],
      "_key__3": "$.*?/|\\^(){}+@[]&_-"
    }
  ]
}
jq - commandline JSON processor [version 1.6]

Usage:  jq [options] <jq filter> [file...]
        jq [options] --args <jq filter> [strings...]
        jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

        $ echo '{"foo": 0}' | jq .
        {
                "foo": 0
        }

Some of the options include:
  -c               compact instead of pretty-printed output;
  -n               use `null` as the single input value;
  -e               set the exit status code based on the output;
  -s               read (slurp) all inputs into an array; apply filter to it;
  -r               output raw strings, not JSON texts;
  -R               read raw strings, not JSON texts;
  -C               colorize JSON;
  -M               monochrome (don't colorize JSON);
  -S               sort keys of objects on output;
  --tab            use tabs for indentation;
  --arg a v        set variable $a to value <v>;
  --argjson a v    set variable $a to JSON value <v>;
  --slurpfile a f  set variable $a to an array of JSON texts read from <f>;
  --rawfile a f    set variable $a to a string consisting of the contents of <f>;
  --args           remaining arguments are string arguments, not files;
  --jsonargs       remaining arguments are JSON arguments, not files;
  --               terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.
jq - commandline JSON processor [version 1.6]

Usage:  jq [options] <jq filter> [file...]
        jq [options] --args <jq filter> [strings...]
        jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

        $ echo '{"foo": 0}' | jq .
        {
                "foo": 0
        }

Some of the options include:
  -c               compact instead of pretty-printed output;
  -n               use `null` as the single input value;
  -e               set the exit status code based on the output;
  -s               read (slurp) all inputs into an array; apply filter to it;
  -r               output raw strings, not JSON texts;
  -R               read raw strings, not JSON texts;
  -C               colorize JSON;
  -M               monochrome (don't colorize JSON);
  -S               sort keys of objects on output;
  --tab            use tabs for indentation;
  --arg a v        set variable $a to value <v>;
  --argjson a v    set variable $a to JSON value <v>;
  --slurpfile a f  set variable $a to an array of JSON texts read from <f>;
  --rawfile a f    set variable $a to a string consisting of the contents of <f>;
  --args           remaining arguments are string arguments, not files;
  --jsonargs       remaining arguments are JSON arguments, not files;
  --               terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.
jq - commandline JSON processor [version 1.6]

Usage:  jq [options] <jq filter> [file...]
        jq [options] --args <jq filter> [strings...]
        jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

        $ echo '{"foo": 0}' | jq .
        {
                "foo": 0
        }

Some of the options include:
  -c               compact instead of pretty-printed output;
  -n               use `null` as the single input value;
  -e               set the exit status code based on the output;
  -s               read (slurp) all inputs into an array; apply filter to it;
  -r               output raw strings, not JSON texts;
  -R               read raw strings, not JSON texts;
  -C               colorize JSON;
  -M               monochrome (don't colorize JSON);
  -S               sort keys of objects on output;
  --tab            use tabs for indentation;
  --arg a v        set variable $a to value <v>;
  --argjson a v    set variable $a to JSON value <v>;
  --slurpfile a f  set variable $a to an array of JSON texts read from <f>;
  --rawfile a f    set variable $a to a string consisting of the contents of <f>;
  --args           remaining arguments are string arguments, not files;
  --jsonargs       remaining arguments are JSON arguments, not files;
  --               terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.
ok
test_yq_arg_passthrough (__main__.TestYq.test_yq_arg_passthrough) ... {}
jq: --indent takes a number between -1 and 7
Use jq --help for help with command-line options,
or see the jq manpage, or online docs  at https://stedolan.github.io/jq
ok
test_yq_err (__main__.TestYq.test_yq_err) ... /usr/pkg/lib/python3.11/subprocess.py:1125: ResourceWarning: subprocess 15952 is still running
  _warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ok

----------------------------------------------------------------------
Ran 22 tests in 3.059s

OK (expected failures=1)
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='utf-8'>
kislyuk commented 1 year ago

Thank you for your interest in yq. You can determine whether the tests passed via the exit code of the test process. The logging behavior is by design.