kislyuk / yq

Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents
https://kislyuk.github.io/yq/
Apache License 2.0
2.53k stars 81 forks source link

tomlq throws TypeError on documents containing date-time values #160

Closed urdh closed 1 year ago

urdh commented 1 year ago

The TOML loader suffers from the same issue the YAML loader suffered in #5, where it fails to encode datetime values when passing them to jq:

$ tomlq --version
tomlq 3.1.0

$  pip freeze | grep yq
yq==3.1.0

$ tomlq . stuff.toml
tomlq: Error running jq: TypeError: Object of type datetime is not JSON serializable.

parse error: Unfinished JSON term at EOF at line 1, column 22
$ cat stuff.toml
[table]
expires = 9999-12-31T23:59:59.999999

If the output is encoded to something else (e.g. YAML), things work fine:

$ tomlq -y . stuff.toml
table:
  expires: '9999-12-31T23:59:59.999999'

I think this is caused by the omission of the JSONDateTimeEncoder in the non-converting case, which would imply this also affects xq:

--- a/yq/__init__.py      2023-02-15 13:16:35.800556554 +0100
+++ b/yq/__init__.py      2023-02-20 13:03:23.020764028 +0100
@@ -252,12 +252,12 @@
                 import xmltodict
                 for input_stream in input_streams:
                     json.dump(xmltodict.parse(input_stream.read(), disable_entities=True,
-                                              force_list=xml_force_list), jq.stdin)
+                                              force_list=xml_force_list), jq.stdin, cls=JSONDateTimeEncoder)
                     jq.stdin.write("\n")
             elif input_format == "toml":
                 import toml
                 for input_stream in input_streams:
-                    json.dump(toml.load(input_stream), jq.stdin)
+                    json.dump(toml.load(input_stream), jq.stdin, cls=JSONDateTimeEncoder)
                     jq.stdin.write("\n")
             else:
                 raise Exception("Unknown input format")
kislyuk commented 1 year ago

Thanks, fixed. XML does not have native datetime literals that I'm aware of, so should not be affected.