Hay: Code nodes containing blocks don't serialize code correctly

[PossiblyAShrub]

Reproduction (on HEAD):

ysh ysh-0.20.0$ cat error.ysh 
hay define Package
hay define Package/INSTALL

Package {
  name = "osh"
  INSTALL {
    cd dist {
      ./install
    }
  }
}

= _hay()
ysh ysh-0.20.0$ bin/ysh error.ysh 
Traceback (most recent call last):
  File "/home/aolse/oils/oil/bin/oils_for_unix.py", line 198, in <module>
    sys.exit(main(sys.argv))
  File "/home/aolse/oils/oil/bin/oils_for_unix.py", line 170, in main
    return AppBundleMain(argv)
  File "/home/aolse/oils/oil/bin/oils_for_unix.py", line 137, in AppBundleMain
    return shell.Main('ysh', arg_r, environ, login_shell, loader, readline)
  File "/home/aolse/oils/oil/core/shell.py", line 1169, in Main
    cmd_flags=cmd_eval.IsMainProgram)
  File "/home/aolse/oils/oil/core/main_loop.py", line 365, in Batch
    is_return, is_fatal = cmd_ev.ExecuteAndCatch(node, cmd_flags)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 2017, in ExecuteAndCatch
    status = self._Execute(node)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 1816, in _Execute
    status = self._Dispatch(node, cmd_st)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 1549, in _Dispatch
    status = self._DoSimple(node, cmd_st)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 840, in _DoSimple
    status = self._RunSimpleCommand(cmd_val, cmd_st, run_flags)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 520, in _RunSimpleCommand
    run_flags)
  File "/home/aolse/oils/oil/core/executor.py", line 305, in RunSimpleCommand
    return self.RunBuiltin(builtin_i.haynode, cmd_val)
  File "/home/aolse/oils/oil/core/executor.py", line 209, in RunBuiltin
    status = builtin_func.Run(cmd_val)
  File "/home/aolse/oils/oil/builtin/hay_ysh.py", line 414, in Run
    self.cmd_ev.EvalCommand(lit_block.brace_group)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 2093, in EvalCommand
    status = self._Execute(block)  # can raise FatalRuntimeError, etc.
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 1816, in _Execute
    status = self._Dispatch(node, cmd_st)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 1676, in _Dispatch
    status = self._ExecuteList(node.children)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 1893, in _ExecuteList
    status = self._Execute(child)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 1816, in _Execute
    status = self._Dispatch(node, cmd_st)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 1549, in _Dispatch
    status = self._DoSimple(node, cmd_st)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 840, in _DoSimple
    status = self._RunSimpleCommand(cmd_val, cmd_st, run_flags)
  File "/home/aolse/oils/oil/osh/cmd_eval.py", line 520, in _RunSimpleCommand
    run_flags)
  File "/home/aolse/oils/oil/core/executor.py", line 305, in RunSimpleCommand
    return self.RunBuiltin(builtin_i.haynode, cmd_val)
  File "/home/aolse/oils/oil/core/executor.py", line 209, in RunBuiltin
    status = builtin_func.Run(cmd_val)
  File "/home/aolse/oils/oil/builtin/hay_ysh.py", line 395, in Run
    lit_block.lines)
  File "/home/aolse/oils/oil/core/alloc.py", line 67, in SnipCodeBlock
    assert found_left, "Couldn't find left token"
AssertionError: Couldn't find left token
  bin/ysh error.ysh
  ^~~
[ interactive ]:3: errexit PID 49585: command.Simple failed with status 1

[andychu]

Hm is that an old version? 0.20.0 ?

I was not able to reproduce on HEAD

$ cat foo.sh 
hay define Package
hay define Package/INSTALL

Package {
  name = "osh"
  INSTALL {
    cd dist {
      ./install
    }
  }
}

= _hay()

andy@hoover:~/git/oilshell/oil$ ysh foo.sh 
(Dict)   {source: null, children: [{type: "Package", args: [], children: [{type: "INSTALL", args: [], location_str: "foo.sh", location_start_line: 6, code_str: "           "}], attrs: {name: "osh"}}]

If so, not sure when it got fixed ...

[PossiblyAShrub]

I'm not sure if the current HEAD is right either. That code_str is just whitespace.

[andychu]

OK interesting

This area needs another pass, to reflect all the YSH work since Hay was written

• the CODE nodes needs to be serialized to value.Command, which didn't exist
• value.Command might need c.toJson(), and then maybe json write (x) can serialize value.Command nodes -- with an option???

[andychu]

BTW I remembered that we want to get rid of this CODE node convention

Instead of INSTALL { ... } it should just be proc install { }, which is a clearer sign that it's code in the block!