openuado / niet

Parse/Read yaml or json files directly in your shell (sh, bash, ksh, ...)
https://pypi.org/project/niet/
MIT License
39 stars 7 forks source link

read data from stdin or from file and fix CVE-2017-2809 #25

Closed 4383 closed 6 years ago

4383 commented 6 years ago

Hi @dj4ngo!

This pull request introduce a new coolest feature for read data (json/yaml) directly from stdin or from a file.

Examples:

$ echo '{"foo": "bar", "fizz": {"buzz": ["1", "2", "Fizz", "4", "Buzz"]}}' | niet fizz.buzz
1
2
Fizz
4
Buzz
$ echo '{"foo": "bar", "fizz": {"buzz": ["1", "2", "Fizz", "4", "Buzz"]}}' | niet fizz.buzz -f squote
'1' '2''Fizz' '4' 'Buzz'
$ echo '{"foo": "bar", "fizz": {"buzz": ["1", "2", "Fizz", "4", "Buzz"]}}' | niet . -f yaml
fizz:
  buzz:
  - '1'
  - '2'
  - Fizz
  - '4'
  - Buzz
foo: bar

Now for reading from a file you need to pass the filename are the second positional argument:

$ niet . tests/samples/sample.yaml -f json
{
    "project": {
        "meta": {
            "name": "my-project"
        },
        "foo": "bar",
        "list-items": [
            "item1",
            "item2",
            "item3"
        ]
    }
}

The helping command now looks like this:

$ niet --help
usage: niet [-h] [-f {dquote,yaml,ifs,squote,newline,json}] object [file]

Read data from YAML or JSON file

positional arguments:
  object                Path to object separated by dot (.). Use '.' to get
                        whole file. eg: a.b.c
  file                  Optional JSON or YAML filename. If not provided niet
                        read from stdin

optional arguments:
  -h, --help            show this help message and exit
  -f {dquote,yaml,ifs,squote,newline,json}, --format {dquote,yaml,ifs,squote,newline,json}
                        output format

output formats:
  dquote        Add double quotes to result
  yaml  Return object in YAML
  ifs   Return all elements of a list separated by IFS env var
  squote        Add single quotes to result
  newline       Return all element of a list in a new line
  json  Return object in JSON

Also this pull request resolve an exploitable vulnerability (CVE-2017-2809) due to the usage of yaml.load instead of yaml.safe_load

Examples of usages of this vulnerability:

$ echo '!!python/object/apply:os.system ["echo powned; uname -a"]' | niet . -f yaml
powned
Linux vm 4.8.0-54-generic #57~16.04.1-Ubuntu SMP Wed May 24 16:22:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Invalid file. Only support valid json and yaml files
$ echo '!!python/object/apply:os.system ["echo powned; uname -a"]' > /tmp/powned.yaml
$ niet . /tmp/powned.yaml
powned
Linux vm 4.8.0-54-generic #57~16.04.1-Ubuntu SMP Wed May 24 16:22:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Invalid file. Only support valid json and yaml input

Examples with the fix:

$ echo '!!python/object/apply:os.system ["echo powned; uname -a"]' | niet . -f yaml
Invalid code injection! :)
could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:os.system'
  in "<unicode string>", line 1, column 1:
    !!python/object/apply:os.system  ...
    ^
Invalid file. Only support valid json and yaml files

Fix #12