itchyny / gojq

Pure Go implementation of jq
MIT License
3.3k stars 119 forks source link

Import statement for importing the contents of a file #221

Closed stereobutter closed 12 months ago

stereobutter commented 1 year ago

Context

At work we use (go)jq as a ~programming~ configuration language i.e. we have a bunch of .jq files versioned in git that are used to render the configuration for some system. Some of these systems embed (base64-encoded) files in their configuration.

Feature request

Add a new import statement that lets users import the contents of a file as string. This would be equivalent to --rawfile name file but usable from a .jq patch file instead of as a command line argument.

# config.ini
[foo]
answer=42
# some_patch.jq
import "config.ini" as &file
.inline_ini_file = (&file | @base64)  # &file == "[foo]\nanswer=42"
itchyny commented 1 year ago

Although this is an interesting request, I strongly suggest submitting to the jq repository. I'm hesitant to implement this feature in gojq until the proposal is accepted in jq, because it might be implemented in different syntax, or they might point out some critical issues.

pkoppstein commented 1 year ago

For further context, please note that jq does allow importing of JSON via an "import" directive:

import "foo" as $foo;

This will read foo.json (as JSON) into $foo.

The existence of this import facility will no doubt complicate enhancements, if only because it raises the question of whether the proposed change should be implemented as some kind of extension or variations of import or of input.

Certainly I hope people like @itchyny will provide some guidance!

wader commented 1 year ago

@stereobutter note that JSON strings are not binary safe as implementations might treat invalid UTF-8 differently:

$ jq -rn --rawfile f <(echo -en '\xff\x01') '$f | @base64' | base64 -d | hexdump -C
00000000  ef bf bd 01                                       |....|
00000004
$ gojq -rn --rawfile f <(echo -en '\xff\x01') '$f | @base64' | base64 -d | hexdump -C
00000000  ff 01                                             |..|
00000002
itchyny commented 12 months ago

Closing as unplanned, but revisit when jq implements this feature.