search

janet-lang / jpm

Janet Project Manager
MIT License
65 stars 21 forks source link

Despite errors, `jpm test` prints `All tests passed.` and has error code 0 #47

Closed saikyun closed 2 years ago

saikyun commented 2 years ago

Using either:

jpm test seems to always return 0.

Example:

git clone https://github.com/saikyun/jpm-wrong-error-code
cd jpm-wrong-error-code
jpm test
running test/error.janet ...
error: wat
  in _thunk [test/error.janet] (tailcall) on line 1, column 1
running test/exit.janet ...
All tests passed.

The tests are running (error "wat") and (os/exit 1).

sogaiu commented 2 years ago

I was able to reproduce this in a Windows environment using jpm at the latest commit and janet at the latest commit.

On a side note, behavior was as expected under a Linux environment -- i.e.

$ jpm test
running test/error.janet ...
error: wat
  in _thunk [test/error.janet] (tailcall) on line 1, column 1
non-zero exit code in test/error.janet: 1
running test/exit.janet ...
non-zero exit code in test/exit.janet: 1
Failing test scripts: 2
bakpakin commented 2 years ago

Looks like this was an error in Janet, has been fixed on master: https://github.com/janet-lang/janet/commit/427f7c362e19117c6ac91b9fe2419019c0adbeb3

We were using the wrong member of a structure to get the return value of the process.

sogaiu commented 2 years ago

I still get similar errors. May be I didn't update appropriately.

Perhaps @saikyun wouldn't mind trying?

bakpakin commented 2 years ago

I can confirm the update works - tested with latest master and spork with a fake bug introduced.

sogaiu commented 2 years ago

Confirmed it's working here. I think I misinterpreted the output before. Sorry about that.