search

Blacksmoke16 / oq

A performant, and portable jq wrapper to facilitate the consumption and output of formats other than JSON; using jq filters to transform the data.
https://blacksmoke16.github.io/oq/
MIT License
190 stars 15 forks source link

Colon in key breaks yaml parsing #69

Closed LoganBarnett closed 3 years ago

LoganBarnett commented 3 years ago

I'll start by saying I don't know for certain that this is an issue with oq. I've gone looking through the YAML 1.2 spec and it feels ambiguous to me. Though I will admit I don't fluently speak tokenizer yet.

My understanding is it's a common thing in Ruby to parse YAML files with the keys prefixed with a colon. An example is :foo: bar. The reason for doing this is that standard Ruby YAML parsing will pull in the key as :foo, and :foo is a "Symbol" in Ruby - an immutable string. Their String primitive is actually mutable. I am not defending this choice.

When I attempt to parse such a document with oq this is what I get:

oq '.foo' <<< ":foo: bar"
parse error: Expected string key before ':' at line 1, column 1

I did find an online YAML parser, and it supports the prefixed colon. You can see an example payload here. This parser is written in Python instead of Ruby. I see these two languages as very close cousins, and I am open to the possibility that they are accomplices to the crime of deviation from the spec.

It would be nice if someone who speaks tokenizer (or format spec) more fluently than me to confirm that Ruby and Python are supporting unsupported documents, or if oq needs to support the colon present in the key. I did find something indicating a key is denoted by some legal string, a colon, and a space (the space is required). This would make foo:bar illegal as a key value indication. oq treats these illegally, as it should. So perhaps the colon-as-prefix is a special case?

At the very least perhaps someone else will happen upon this, and it will save them research + ticket writing.

Thanks for maintaining a wonderful tool!

Blacksmoke16 commented 3 years ago

@LoganBarnett I think you meant to use oq -i yaml '.foo' <<< ":foo: bar". Without -i yaml it'll try to parse the input as JSON, of which that isn't valid JSON.

Similarly this replicates using jq directly as well:

jq '.foo' <<< ":foo: bar"
parse error: Expected string key before ':' at line 1, column 1
LoganBarnett commented 3 years ago

@Blacksmoke16 wow, my deepest apologies! It's been that kind of day :( Thank you for a polite reply despite the obvious miss.