Unusual behaviour with empty YAML documents

[bobbwest]

Environment:

$ oq --version
jq: jq-1.6
oq: 1.2.1
$ uname
Linux

oq outputs a single null value when presented with empty YAML input; this is consistent with jq -n, although not consistent with how jq treats empty input:

$ oq -i yaml -o json . </dev/null
null 
$ jq . </dev/null

This causes further issues when trying to perform some function on empty input (you might not know in advance whether the file is empty or not, or it might be generated from some other pre-processing step). I think this should simply return an empty document, like jq does:

$ oq -i yaml -o json '.[]' </dev/null
jq: error (at <stdin>:0): Cannot iterate over null (null)
$ jq '.[]' </dev/null

There are also issues (perhaps unrelated) when jq returns nothing and yaml output is requested.

# this works when an object is returned:
$ oq -i yaml -o yaml '.[] | select(.name == "foo")' <<<$'- name: foo'
--- 
name: foo

# this fails when no object is returned:
$ oq -i yaml -o yaml '.[] | select(.name != "foo")' <<<$'- name: foo'
oq error: Unexpected token: <EOF> at line 1, column 1

# but it is fine when output as JSON
$ oq -i yaml -o json '.[] | select(.name != "foo")' <<<$'- name: foo'

I accept that the representation (or validity) of an empty document is not necessarily the same for YAML and JSON, and the behaviour of jq -n doesn't help clarify things.

Any thoughts on this?

[Blacksmoke16]

I'm pretty that the YAML spec makes it so that an empty file is represented as null, which is why that's what you get back. This behavior can also be seen using Python's yaml module:

#!/usr/bin/env python

import yaml

with open("test.yml", 'r') as stream:
    print(yaml.safe_load(stream)) # => None

I think this should simply return an empty document, like jq does:

Wouldn't this be a good use case for the .[]? filter?

$ oq -i yaml -o json '.[]?' </dev/null

$ echo '[1,2,3]' | oq -i yaml -o json '.[]?'
1
2
3

There are also issues (perhaps unrelated) when jq returns nothing and yaml output is requested.

Thanks, I'll check this out.

[Blacksmoke16]

@bobbwest So the issue when jq returns nothing is actually part a Crystal bug and part an oq bug. The oq side of things will be handled via https://github.com/Blacksmoke16/oq/pull/83. However that depends on https://github.com/crystal-lang/crystal/pull/10864.

Crystal will be doing a release soon, however I'm not sure if this'll make it into it or not. Either way I'll be sure to do another release when the fix is included upstream so that the binary doesn't have this bug. In the meantime you can build the binary manually if you really need a quick fix.

[bobbwest]

Thanks, much appreciated. oq is a great tool.

[Blacksmoke16]

Unfortunately the upstream fix didn't make it into this latest release and the next one won't be for a few months. Probably going to update that PR to handle it manually in code and leave a TODO to remove it later.