databricks / sjsonnet

Apache License 2.0
267 stars 55 forks source link

Inconsistency between sjsonnet and jsonnet #144

Closed ieluw closed 2 years ago

ieluw commented 2 years ago

I happened to use jsonnet to compile Runbot's jsonnet file, which was failing but passed for sjsonnect.

The command I used:

/usr/local/Cellar/jsonnet/0.18.0/bin/jsonnet /Users/Lei.Wu/databricks/universe/ci/runbot/staging/runbot-app.jsonnet

and the error message from jsonnet is:

RUNTIME ERROR: couldn't manifest function in JSON output.
    std.jsonnet:36:49   function <anonymous>
    std.jsonnet:592:9-26    function <format_code>
    std.jsonnet:768:15-50   thunk <s>
    std.jsonnet:773:24  thunk <str>
    std.jsonnet:480:30-33
    std.jsonnet:480:19-34   thunk <w>
    std.jsonnet:476:11  thunk <w>
    std.jsonnet:472:12  function <aux>
    std.jsonnet:476:7-17    function <padding>
    std.jsonnet:480:7-38    function <pad_left>
    ...
    std.jsonnet:774:11-60   function <format_codes_obj>
    std.jsonnet:774:11-60   function <format_codes_obj>
    std.jsonnet:779:7-43    function <anonymous>
    std.jsonnet:237:7-23    function <anonymous>
    /Users/Lei.Wu/databricks/universe/ci/runbot/staging/../builds/dust/utils.jsonnet.TEMPLATE:(168:5)-(188:6)   thunk <array_element>
    /Users/Lei.Wu/databricks/universe/ci/runbot/staging/../builds/dust/testshard-jobs.jsonnet.TEMPLATE:186:28-46    thunk <array_element>
    /Users/Lei.Wu/databricks/universe/ci/runbot/staging/../builds/dust/testshard-jobs.jsonnet.TEMPLATE:(185:22)-(189:6) object <anonymous>
    /Users/Lei.Wu/databricks/universe/ci/runbot/staging/runbot-app.jsonnet.TEMPLATE:(187:5)-(203:6) thunk <array_element>
    /Users/Lei.Wu/databricks/universe/ci/runbot/staging/runbot-app.jsonnet.TEMPLATE:(16:11)-(901:4) object <anonymous>
    During manifestation
szeiger commented 2 years ago

This is a bug in the Jsonnet files. They pass a function value to a %s interpolation which is supposed to fail:

$ echo '"%1s" % function() "foo"' | jsonnet -
RUNTIME ERROR: couldn't manifest function in JSON output.
    std.jsonnet:36:49   function <anonymous>
    std.jsonnet:592:9-26    function <format_code>
    std.jsonnet:722:15-60   thunk <s>
    std.jsonnet:727:24  thunk <str>
    std.jsonnet:480:30-33
    std.jsonnet:480:19-34   thunk <w>
    std.jsonnet:476:11  thunk <w>
    std.jsonnet:472:12  function <aux>
    std.jsonnet:476:7-17    function <padding>
    std.jsonnet:480:7-38    function <pad_left>
    std.jsonnet:727:15-39   thunk <s_padded>
    std.jsonnet:733:55-63   thunk <v>
    std.jsonnet:733:11-64   function <format_codes_arr>
    std.jsonnet:733:11-64   function <format_codes_arr>
    std.jsonnet:781:7-48    function <anonymous>
    std.jsonnet:237:7-23    function <anonymous>
    <stdin>:1:1-25

Sjsonnet erroneously accepts this and silently evaluates a nullary function, just like in https://github.com/databricks/sjsonnet/issues/120.

szeiger commented 2 years ago

The specification defers to Python for std.format, which shows yet another behavior:

$ python -c '"%1s" % (lambda : "foo")'

The function value is accepted and converted to a string, but it's an empty string, the function is not evaluated.

szeiger commented 2 years ago

The error message by jsonnet is misleading. Functions are disallowed as values in std.format but they can be manifested:

$ echo "function() 42" | jsonnet -
42