code snippets in error messages should be terminal-escaped

[andychu]

"A fuzz test"

Should not hit this, should hit an error message

$ _bin/cxx-dbg/osh_eval _bin/cxx-opt/osh_eval 
osh_eval: _gen/bin/osh_eval.mycpp.cc:33251: Tuple2<syntax_asdl::word_t*, bool> word_parse::WordParser::_ReadWord(types_asdl::lex_mode_t): Assertion `0' failed.
Aborted (core dumped)

[IamAbhinav03]

Hi👋 , I would like to work on this issue. Can I get more information on this issue?

[andychu]

Hi, thanks for the message! Are you able to get

_bin/cxx-dbg/osh_eval running, as on this page?

https://github.com/oilshell/oil/wiki/Oil-Native-Quick-Start

If so then the repo is just to run itself as a shell script :)

_bin/cxx-dbg/osh_eval  _bin/cxx-dbg/osh_eval 

I didn't look into the reason why, but it would nice to see a GDB stack trace

Thank you

[IamAbhinav03]

I tried installing this in my kali virtual machine and I met with an "Unmet dependencies" error when trying to run "build/py-sh --ubuntu-deps". I looked into the py.sh and saw it was using python-dev so I changed it to python2-dev. Even after that, I'm met with more Unmet dependencies errors. I tried sudo apt --fix-broken install, but it doesn't seem to help. I'm inserting the screenshot of the error I got. Can you help me?

[andychu]

Hm I'm not familiar with Kali ... Maybe you can install or ugprade libc6 manually and see if it changes the errors?

Either way I'm sure there is some way to get cmake / ninja / etc. on the machine

That function is mainly meant for Ubuntu users, as an example

I thought we had one for Alpine Linux too

[IamAbhinav03]

I fixed the error before and followed all other steps. I was following this https://github.com/oilshell/oil/blob/master/mycpp/README.md to oil-native working, but when I ran oil$ deps/from-git.sh mypy-pip-install I got ImportError: cannot import name 'BinaryIO' from 'typing'

Is there anything I can do to fix it?

[andychu]

What this is trying to do is install the dependencies of MyPy test-requirements.txt on your machine

https://github.com/oilshell/oil/blob/master/deps/from-git.sh#L57

So maybe you can run those steps manually and see what breaks? We need a more detailed log

One issue I see is that maybe-our-python3 can run either the Python 3.10 we build, or your system Python 3

And depending on which one, maybe you get the ImportError

It would be nice to know which path maybe-our-python3 takes on your machine -- did the Python 3.10 build succeed?

Maybe we should fix it so there is only one code path

[IamAbhinav03]

Hi. I fixed those errors. It was due to my dubm mistake. I thought the commands in https://github.com/oilshell/oil/blob/master/mycpp/README.md should be run in the oil shell due to the oil$ before the code. I fixed all of it and now it runs and I recreated the error, but I have a doubt. When I ran the command in the first post (https://github.com/oilshell/oil/issues/1326#issue-1384496109) I got this error msg osh: Couldn't open '_bin/cxx-opt/osh_eval': Bad file descriptor and when I ran the _bin/cxx-dbg/osh_eval _bin/cxx-dbg/osh_eval mentioned in https://github.com/oilshell/oil/issues/1326#issuecomment-1285879428 I got the same error you described. What is the difference between those commands, I didn't quite understand what those commands do.

[andychu]

osh: Couldn't open '_bin/cxx-opt/osh_eval'

Does that file exist? You can build either opt or dbg, but it should be able to read it in both Python and C++

I would be very curious to see the difference, that could definitely help narrow the bug down

[IamAbhinav03]

There is a folder named cxx-opt inside _bin. But it contains another folder called mycpp; inside it, another folder called examples having a bunch of mycpp files. There is no file called osh_eval inside _bin/cxx-opt/

[andychu]

You can build that file with Ninja

But why not just run on cxx-dbg? The idea is to pass a "junk binary" file to OSH as the shell script

[AtomsForPeace]

I ran the following and got a different looking error than what you posted originally:

➜ oil git:(master) ✗ _bin/cxx-asan/osh _bin/cxx-asan/osh ,"m %xxNNddd88X"yX2yX2yxUyxeyxey8880hhhDDStd8880PtdqqqQtdRtdX"yX2yX2y==/lib64/ld-linux-x86-64.so.2 GNUGNUO2KδuNU f 0BB ^ '_bin/cxx-asan/osh':1: '\x7fELF\x02\x01\x01\x03\x01}J\x87\x9eR\xe5\x0c\x08E\xdd\x95u1\x904\xc5\xb9\x9c@8\xf2\x8b\x1c2\x90Vd\xc5\x89\x05\x90\xe6N\x08\x97\xe5\xab\x90L\xda*\x90sE\xc2\x1d\xd2' not found pkemBhtVgUaOgh OG_2h(L CY5|?LoY@}Vt ~x'W n3|5

                                                    Br 
                                                       (4RgGIAa:u+
                                                                  ^M'~G
                                                                       FG9

                                             ^

'_bin/cxx-asan/osh':4: Unexpected left paren (might need a space before it)

[andychu]

Oh interesting! Hm actually this is not terrible, at least it doesn't crash. Very similar output from zsh and dash:

$ dash /bin/dash
/bin/dash: 1: Syntax error: word unexpected (expecting ")")

andy@hoover:~/git/oilshell/oil$ bash /bin/bash
/bin/bash: /bin/bash: cannot execute binary file

andy@hoover:~/git/oilshell/oil$ zsh /bin/zsh
/bin/zsh:1: bad pattern: ^@^@^@^@^@^@^P^@^@^@^@^@^@^A^@^@^@^D^@^@^@^@^P^K^@^@^@^@^@^@^P^K^@^@^@^@^@^@^P^K^@^@^@^@^@\M-p\M-!^A^@^@^@^@^@\M-p\M-!^A^@^@^@^@^@^@^P^@^@^@^@^@^@^A^@^@^@^F^@^@^@\M-p\M-:^L^@^@^@^@^@\M-p\M-J^L^@^@^@^@^@\M-p\M-J^L^@^@^@^@^@\M-Xq^@^@^@^@^@^@\M-p\M-(^A^@^@^@^@^@^@^P^@^@^@^@^@^@^B^@^@^@^F^@^@^@(\M-G^L^@^@^@^@^@(\M-W^L^@^@^@^@^@(\M-W^L^@^@^@^@^@ ^B^@^@^@^@^@^@ ^B^@^@^@^@^@^@^H^@^@^@^@^@^@^@^D^@^@^@^D^@^@^@8^C^@^@^@^@^@^@8^C^@^@^@^@^@^@8^C^@^@^@^@^@^@ ^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^H^@^@^@^@^@^@^@^D^@^@^@^D^@^@^@X^C^@^@^@^@^@^@X^C^@^@^@^@^@^@X^C^@^@^@^@^@^@D^@^@^@^@^@^@^@D^@^@^@^@^@^@^@^D^@^@^@^@^@^@^@S\M-etd^D^@^@^@8^C^@^@^@^@^@^@8^C^@^@^@^@^@^@8^C^@^@^@^@^@^@ ^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^H^@^@^@^@^@^@^@P\M-etd^D^@^@^@t\M-'^K^@^@^@^@^@t\M-'^K^@^@^@^@^@t\M-'^K^@^@^@^@^@\M-$!^@^@^@^@^@^@\M-$!^@^@^@^@^@^@^D^@^@^@^@^@^@^@Q\M-etd^F^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^P^@^@^@^@^@^@^@R\M-etd^D^@^@^@\M-p\M-:^L^@^@^@^@^@\M-p\M-J^L^@^@^@^@^@\M-p\M-J^L^@^@^@^@^@^P^U^@^@^@^@^@^@^P^U^@^@^@^@^@^@^A^@^@^@^@^@^@^@/lib64/ld-linux-x86-64.so.2^@^@^@^@^@^D^@^@^@^P^@^@^@^E^@^@^@GNU^@^B\M-^@^@\M-@^D^@^@^@^A^@^@^@^@^@^@^@^D^@^@^@^T^@^@^@^C^@^@^@GNU^@5\M-1$\M-x\M-^X\M-(
/bin/zsh:1: command not found: ^L^E\M-;\M- 
/bin/zsh:12: parse error near `)'

[andychu]

I think the bug in OSH is that the code quotations should be terminal-escaped ? That should be pretty easy

(I'm working on "J8 Notation" which is like JSON, but can represent arbitrary binary data in a readable way)

The filename is already escaped, but the code quotation isn't.

Thanks for trying it!

---

Another possibility is to enforce that the Oils source code is valid UTF-8 , and does not contain ASCII control chars, which is the rule that JSON itself follows.

That is probably too restrictive since people do put binary data in here docs and single quoted strings!