Add Pony

[shyba]

Please complete the following information about the language:

• Name: Pony
• Website: https://www.ponylang.io/
• Language Version: 0.38.3

The following are optional, but will help us add the language:

• Test Frameworks: PonyTest https://stdlib.ponylang.io/ponytest-PonyTest/
• How to install: https://www.ponylang.io/learn/#installing-pony
• How to compile/run: ponyc generates a binary
• Any comments: (e.g., what's interesting about this language) It is a compiled, safe, actor model based language. Pony uses a capabilities model for memory safety and offers strong guarantees on concurrency, making it data-race free.

On codewars, it would be great to use Pony to get challenged over its capabilities system and exercise new ways to organize code so it fits this model.

---

• [x] Installing (https://github.com/ponylang/ponyc/blob/main/INSTALL.md#linux)
• [x] Compiling and executing (compile with ponyc)
• [ ] Testing PonyTest
• [ ] Test Output
• [ ] Editor (CodeMirror mode)
• [ ] Icon

---

:+1: reaction might help to get this request prioritized.

[kazk]

Is the output of PonyTest customizable? If not, does it support machine-readable format like JSON? See https://github.com/codewars/runner/blob/main/docs/messages.md for what we need.

[hideki-lu]

Is the output of PonyTest customizable? If not, does it support machine-readable format like JSON? See https://github.com/codewars/runner/blob/main/docs/messages.md for what we need.

Pony have these two JSON parsers I guess can be useful https://github.com/jemc/pony-jason and https://github.com/niclash/jay

[kazk]

No, I wasn't asking if Pony supports JSON or not.

Does PonyTest have customizable output? Does PonyTest have any machine-readable output formats (such as JSON) as an option?

[shyba]

Hi,

This is how the output looks like: https://gist.github.com/shyba/4cb1e456a4ab32ef2e74c3d06e8e6805 That shows a failing test, a passing one, an error and something being logged.

Unfortunately, the output is hardcoded here: https://github.com/ponylang/ponyc/blob/main/packages/ponytest/_test_record.pony#L31-L36 And there is no JSON output option.

---

I have it cloned locally and I am trying to make it customizable, but it will need to go through a feature request and a pull request.

IMO, the current options are:

• parse the output
• copy ponytest into a temporary package and make it customizable until the official gets the feature

If parsing the output isn't good enough, ping me and I will try to get a fork with the customizable output up.

---

I will also check how to add pony to code mirror. I can't find it on their list of supported langs.

[kazk]

I guess we only need the following section:

**** FAILED: addition
/home/user/github/study/pony/tutorial/ponytest/main.pony:20: Assert eq passed.  Got (4) == (4)
Test threw an error
Short test finished
Tearing down test
**** FAILED: subtraction
/home/user/github/study/pony/tutorial/ponytest/main.pony:28: Assert eq failed.  Expected (3) == (2)
Short test finished
Tearing down test
---- Passed: correct subtraction
/home/user/github/study/pony/tutorial/ponytest/main.pony:34: Assert eq passed.  Got (2) == (2)
Short test finished
Tearing down test
---- Passed: logs something
/home/user/github/study/pony/tutorial/ponytest/main.pony:40: Assert eq passed.  Got (2) == (2)
something
Short test finished
Tearing down test
----

Extracting

Each test case can be extracted with

/^((?:-{4} Passed|\*{4} FAILED)[^]+?(?=-{4}|\*{4}))/gm

assuming we always have one of the following format for each case (... are placeholders).

---- Passed: xxxx
...
...
****

---- Passed: xxxx
...
...
----

**** FAILED: xxxx
...
...
****

**** FAILED: xxxx
...
...
----

Extracted test cases will be either

---- Passed: xxxx
...
...

or

**** FAILED: xxxx
...
...

Transforming Passed

The passed test will extract the test case name, remove unnecessary lines and add <PASSED::>.

<IT::>xxxx

<PASSED::>Test Passed

<COMPLETEDIN::>

Transforming Failed

The failed case will be

<IT::>xxxx

<FAILED::>...

<COMPLETEDIN::>

or

<IT::>xxxx

<ERROR::>...

<COMPLETEDIN::>

So,

**** FAILED: subtraction
/home/user/github/study/pony/tutorial/ponytest/main.pony:28: Assert eq failed.  Expected (3) == (2)
Short test finished
Tearing down test

should become

<IT::>subtraction

<FAILED::>Assert eq failed.  Expected (3) == (2)

<COMPLETEDIN::>

**** FAILED: addition
/home/user/github/study/pony/tutorial/ponytest/main.pony:20: Assert eq passed.  Got (4) == (4)
Test threw an error
Short test finished
Tearing down test

should become

<IT::>addition

<PASSED::>Assert eq passed.  Got (4) == (4)

<ERROR::>Test threw an error

<COMPLETEDIN::>

Thoughts

It's not great, but doable.

@shyba What's Short test finished? Is there any other special messages from the PonyTest (e.g., Long test finished)? I'll need the list so that I can preserve the outputs from the solution.

Is the assertion result always Assert eq?

[shyba]

What's Short test finished? Is there any other special messages from the PonyTest (e.g., Long test finished)? I'll need the list so that I can preserve the outputs from the solution.

Short test finished comes from https://github.com/ponylang/ponyc/blob/7f1f475187dc31d411a6770789960008bdc4f8ce/packages/ponytest/_test_runner.pony#L188 It means the test is fast, but I need to examine closely how it decides what is short or long. There are a couple more messages on that file. I will prepare a list and try to hit them over examples. There should be a way to control timeouts as well.

Those extra lines are caused by using --verbose option. Output without it looks like:

1 test started, 0 complete: subtraction started
2 tests started, 0 complete: correct subtraction started
3 tests started, 0 complete: logs something started
4 tests started, 0 complete: addition started
4 tests started, 1 complete: logs something complete
4 tests started, 2 complete: subtraction complete
4 tests started, 3 complete: correct subtraction complete
4 tests started, 4 complete: addition complete
**** FAILED: addition
Test threw an error
**** FAILED: subtraction
/home/user/github/study/pony/tutorial/ponytest/main.pony:28: Assert eq failed.  Expected (3) == (2)
---- Passed: correct subtraction
---- Passed: logs something
----
---- 4 tests ran.
---- Passed: 2
**** FAILED: 2 tests, listed below:
**** FAILED: addition
**** FAILED: subtraction

That's cleaner, but test logs are gone. This is why I went to the --verbose route, but maybe there is a way to output user logs without --verbose.

Is the assertion result always Assert eq?

There are other kinds of assertions: https://github.com/ponylang/ponyc/blob/7f1f475187dc31d411a6770789960008bdc4f8ce/packages/ponytest/test_helper.pony#L52

They seem to log different messages as well. Maybe we can try to parse after the <file>:<line_number> pattern right?

Thank you a lot for looking into this. I will check logging and try to hit the other cases soon.