Add J - Githubissues

RustemB commented 4 years ago

Name: J
Website: https://www.jsoftware.com/
Language Version: j807, j901
Test Frameworks: just load file/script and use needed functions

How to install:

# for j901
$ wget http://www.jsoftware.com/download/j901/install/j901_amd64.deb
$ sudo dpkg -i j901_amd64.deb
# for j807
$ wget http://www.jsoftware.com/download/j807/install/j807_amd64.deb
$ sudo dpkg -i j807_amd64.deb

[x] Installing (see above)
[x] Executing (jconsole)
[ ] Testing
[x] Editing. CodeMirror mode. (J mode for CM 4.2)
[x] Icon (https://github.com/file-icons/icons/blob/master/svg/J.svg)

Steffan153 commented 4 years ago

This would be interesting, but I think it would be tricky to get a working setup.

g964 commented 4 years ago

The author will have to write tests using some J primitives since there are no testing framework. CW could concatenate the solution and the tests and run the whole but I know nothing about the CW runner. I wrote something from the "multiply" kata for J :

multiply =: 4 : 0
   x * y
)

testing =: 3 : 0
   t =. (actual = expect)
   if. t = 0 do. 
        echo 'Incorrect value, expected is: '
        echo expect
        echo 'but got: '
        echo actual
   else.
      echo 'correct'
      echo 'actual'
      echo actual
      echo 'expect'
      echo expect
   end.
   assert. t = 1
)
fixedtests =: 3 : 0 
   actual =: 5 multiply 8
   expect =: 40
   testing ''
   echo '---------------'
   actual =: 8 multiply 9
   expect =: 72
   testing ''
   echo '---------------'
   actual =: 10 multiply 1000
   expect =: 9999
   testing ''
   echo 'End of fixed tests'
)

fixedtests ''

The last test fails and tests are stopped without echoing 'End of fixed tests'. The solution part is the function "multiply" and after that the tests part. You can try it at "tio" with the first two parts in "code" and the "fixedtests ''" part in input. You get the result in "output".

RustemB commented 4 years ago

There are many other ways to produce test units

g964 commented 4 years ago

@RustemB: could be good if you gave something with details:-)

Steffan153 commented 4 years ago

We could write our own test framework and just include it, right?

kazk commented 4 years ago

Yes, anything should work as long as it produces the Codewars output. We should use the existing tools when possible though. How does J developers write tests?

Bubbler-4 commented 4 years ago

How does J developers write tests?

Usually it's done using the built-in assertion primitive assert., but on assertion failure it gives a fixed output:

assert. 1 = 0
|assertion failure: examplePgm
|   1    =0

(And it can be turned off using system functions.)

So I think we will need a custom test framework. But we have a special challenge here because J does not allow multi-line statements or nesting multi-line definitions. Do we have other languages with similar constraints?

kazk commented 4 years ago

I found this: https://github.com/jsoftware/general_unittest

Can we fork that and customize the test output? https://github.com/jsoftware/general_unittest/blob/c0916768bba3832fbb9bf2305d34575a5adc0ad0/unittest.ijs#L21-L62

But I guess we can only produce top level assertion results without test names with that.

So I think we will need a custom test framework. But we have a special challenge here because J does not allow multi-line statements or nesting multi-line definitions. Do we have other languages with similar constraints?

Can you elaborate? Do you mean it's not easy to have the test name and the test duration because of J's limitation?

Bubbler-4 commented 4 years ago

I thought it's not easy to visually simulate how the test cases are written in other languages. That unittest library does look promising (though it might still need a complete rewrite for our needs, it's indeed worth taking a look).

mlliarm commented 2 years ago

Hello !

I'm not that well versed with J yet (currently learning APL), but I've found the following information about testing J programs:

https://code.jsoftware.com/wiki/Addons/Test_scripts
https://code.jsoftware.com/wiki/Addons/general/unittest (adds more information on the unittest Github repository you've already found, plus it has some examples of its use)

Hopefully people more knowledgeable than me in J will jump in and help, as it seems to me that the APL/j/k/... (array lang.) community just took notice of this feature request.

Cheers !

Bubbler-4 commented 2 years ago

With the newly introduced {{...}} notation, I think I've got some nice way to write down tests. (Which makes this limited to J900 or higher, but I don't think J80x support is necessary after all.)

FmtMsg =: (LF;'<:LF:>')&stringreplace

Display =: {{
  'type message' =. y
  echo LF,'<', (toupper type), '::>', FmtMsg message
}}

FmtValue =: {{
  s =. ": y
  ([: , ,.&LF)^:(<:#$s) s
}}

NB. [failure message] Expect value
NB. 0 is treated as failure, anything else OK
Expect =: 'Value is not what was expected'&$: : {{
  'failmsg value' =. x;y
  {{
    Display 'PASSED';'Test Passed'
  }} ` {{
    Display 'FAILED';y
  }} @. (0=y) failmsg
}}

NB. [failure message] AssertEq (actual ; expected)
NB. Use ,&< in place of ; if either side may be boxed
AssertEq =: {{
  'actual expected' =. y
  msg =. (FmtValue actual), ' should equal ', FmtValue expected
  msg Expect actual -:!.0 expected
  :
  'actual expected' =. y
  msg =. x, ': ', (FmtValue actual), ' should equal ', FmtValue expected
  msg Expect actual -:!.0 expected
}}

NB. [failure message] AssertNotEq (actual ; expected)
NB. Use ,&< in place of ; if either side may be boxed
AssertNotEq =: {{
  'actual expected' =. y
  msg =. (FmtValue actual), ' should not equal ', FmtValue expected
  msg Expect -. actual -:!.0 expected
  :
  'actual expected' =. y
  msg =. x, ': ', (FmtValue actual), ' should not equal ', FmtValue expected
  msg Expect -. actual -:!.0 expected
}}

NB. [failure message] AssertApproxEq (actual ; expected)
NB. Use ,&< in place of ; if either side may be boxed
NB. J has tolerant comparison built-in, but it cannot be set higher than 2^_34
AssertApproxEq =: {{
  'actual expected' =. y
  msg =. (FmtValue actual), ' should approximately equal ', FmtValue expected
  msg Expect actual -:!.(2^_34) expected
  :
  'actual expected' =. y
  msg =. x, ': ', (FmtValue actual), ' should approximately equal ', FmtValue expected
  msg Expect actual -:!.(2^_34) expected
}}

Describe =: {{
  Display 'DESCRIBE' ; u
  starttime =. 6!:9 ''
  v ''
  endtime =. 6!:9 ''
  elapsed =. (endtime - starttime) % 6!:8 ''
  Display 'COMPLETEDIN' ; 0j6 ": elapsed
}}

It =: {{
  Display 'IT' ; u
  starttime =. 6!:9 ''
  v ''
  endtime =. 6!:9 ''
  elapsed =. (endtime - starttime) % 6!:8 ''
  Display 'COMPLETEDIN' ; 0j6 ": elapsed
}}

NB. Example test suite

'Describe' Describe {{
  'It1' It {{
    'True' Expect 1
    'False' Expect 0
    Expect 1
    Expect 0
  }}
  'It2' It {{
    AssertEq 1 ; 1
    AssertEq 1 2 3 ; 1 2 3
    AssertEq 1.414213562373 ; %:2
    AssertApproxEq 1.414213562373 ; %:2
    AssertEq ('abc',:'def') ; ('abc','def',:'ghi')
  }}
}}

Outputs


<DESCRIBE::>Describe

<IT::>It1

<PASSED::>Test Passed

<FAILED::>False

<PASSED::>Test Passed

<FAILED::>Value is not what was expected

<COMPLETEDIN::>0.002855

<IT::>It2

<PASSED::>Test Passed

<PASSED::>Test Passed

<FAILED::>1.41421 should equal 1.41421

<PASSED::>Test Passed

<FAILED::>abc<:LF:>def<:LF:> should equal abc<:LF:>def<:LF:>ghi<:LF:>

<COMPLETEDIN::>0.004143

<COMPLETEDIN::>0.009831

2-space indentation is arbitrary because nested definitions were not allowed by the language, so there is no precedent of nested code blocks like this. Due to the limit in the number of args, I decided to put both data being compared on the right argument as a 2-item boxed array.

kazk commented 2 years ago

I thought it's not easy to visually simulate how the test cases are written in other languages.

We don't need to make it look like other languages. The tests should look natural for the language. I'm totally fine with the syntax of general/unittest if that's what J users are familiar with.

2-space indentation is arbitrary because nested definitions were not allowed by the language, so there is no precedent of nested code blocks like this.

<DESCRIBE::> is optional, so we don't need to support it.

Also, is it possible to use assert? The test to stop at first assertion failure.

tangentstorm commented 2 years ago

Well, I was about to volunteer to write a runner, but @Bubbler-4 beat me to it. :)

version

The {{ .. }} syntax requires j902, but with 903 right around the corner, that's probably the version to target.

codemirror

J comes with a web-based IDE called JHS that includes a codemirror mode:

https://github.com/jsoftware/ide_jhs/blob/master/js/codemirror/j/j.4.2.js

icon

There are several different colors of the same J logo available:

https://code.jsoftware.com/wiki/J_Logos

Bubbler-4 commented 2 years ago

We don't need to make it look like other languages. The tests should look natural for the language. I'm totally fine with the syntax of general/unittest if that's what J users are familiar with.

general/unittest was written ~12 years ago and I don't think it is the right style to write J code today. Apparently 2-space indent was a thing too, mainly for nested control structures.

Also, is it possible to use assert? The test to stop at first assertion failure.

I think I have a way to do it. (That said, I guess it needs to report other errors as test failures. It must be possible through dberr / dberm.) Should both options (run all / stop at first failure) be supported or is supporting only one OK?

Bubbler-4 commented 2 years ago

This seems to work as intended:

ASSERT_THROW =: 0

FmtMsg =: (LF;'<:LF:>')&stringreplace

Display =: {{
  'type message' =. y
  echo LF,'<', (toupper type), '::>', FmtMsg message
}}

FmtValue =: {{
  s =. ": y
  ([: , ,.&LF)^:(<:#$s) s
}}

NB. [failure message] Expect value
NB. 0 is treated as failure, anything else OK
Expect =: 'Value is not what was expected'&$: : {{
  'failmsg value' =. x;y
  {{
    Display 'PASSED';'Test Passed'
  }} ` {{
    Display 'FAILED';y
    assert^:ASSERT_THROW 0
  }} @. (0=y) failmsg
}}

NB. [failure message] AssertEq (actual ; expected)
NB. Use ,&< in place of ; if either side may be boxed
AssertEq =: {{
  'actual expected' =. y
  msg =. (FmtValue actual), ' should equal ', FmtValue expected
  msg Expect actual -:!.0 expected
  :
  'actual expected' =. y
  msg =. x, ': ', (FmtValue actual), ' should equal ', FmtValue expected
  msg Expect actual -:!.0 expected
}}

NB. [failure message] AssertNotEq (actual ; expected)
NB. Use ,&< in place of ; if either side may be boxed
AssertNotEq =: {{
  'actual expected' =. y
  msg =. (FmtValue actual), ' should not equal ', FmtValue expected
  msg Expect -. actual -:!.0 expected
  :
  'actual expected' =. y
  msg =. x, ': ', (FmtValue actual), ' should not equal ', FmtValue expected
  msg Expect -. actual -:!.0 expected
}}

NB. [failure message] AssertApproxEq (actual ; expected)
NB. Use ,&< in place of ; if either side may be boxed
NB. J has tolerant comparison built-in, but it cannot be set higher than 2^_34
AssertApproxEq =: {{
  'actual expected' =. y
  msg =. (FmtValue actual), ' should approximately equal ', FmtValue expected
  msg Expect actual -:!.(2^_34) expected
  :
  'actual expected' =. y
  msg =. x, ': ', (FmtValue actual), ' should approximately equal ', FmtValue expected
  msg Expect actual -:!.(2^_34) expected
}}

Describe =: {{
  Display 'DESCRIBE' ; u
  starttime =. 6!:9 ''
  v ''
  endtime =. 6!:9 ''
  elapsed =. (endtime - starttime) % 6!:8 ''
  Display 'COMPLETEDIN' ; 0j6 ": elapsed
}}

It =: {{
  Display 'IT' ; u
  starttime =. 6!:9 ''
  v :: {{
    errNo =. dberr''
    errMsg =. 'The following error was encountered:',LF,dberm''
    Display^:(errNo~:12) 'FAILED';errMsg
  }} ''
  endtime =. 6!:9 ''
  elapsed =. (endtime - starttime) % 6!:8 ''
  Display 'COMPLETEDIN' ; 0j6 ": elapsed
}}

NB. Example test suite

'Describe' Describe {{
  'It1' It {{
    'True' Expect 1
    'False' Expect 0
    Expect 1
    Expect 0
  }}
  'It2' It {{
    AssertEq 1 ; 1
    AssertEq 1 2 3 ; 1 2 3
    AssertEq 1.414213562373 ; %:2
    AssertApproxEq 1.414213562373 ; %:2
    AssertEq ('abc',:'def') ; ('abc','def',:'ghi')
  }}
  ASSERT_THROW =: 1  NB. stop at first assertion failure
  'It3' It {{
    AssertEq 1 ; 1
    AssertEq 1 2 3 ; 1 2 3
    AssertEq 1.414213562373 ; %:2
    AssertApproxEq 1.414213562373 ; %:2
    AssertEq ('abc',:'def') ; ('abc','def',:'ghi')
  }}
  'It4' It {{
    AssertEq 1 ; 2{0 1  NB. throws index error
    AssertEq 1 ; 1
  }}
}}

kazk commented 2 years ago

@tangentstorm Thanks for the links to the CodeMirror mode and the logo. The CodeMirror mode you linked uses CodeMirror v4.2, but we use v5.61. Hopefully, it still works as is, but it shouldn't be too difficult to update even if it doesn't. For the logo, we need an SVG version, so I'll probably use https://github.com/file-icons/icons/blob/master/svg/J.svg

@Bubbler-4

Should both options (run all / stop at first failure) be supported or is supporting only one OK?

Let's just always stop at the first assertion failure within each test case. I don't think it's necessary to support both.

It =: {{
  Display 'IT' ; u
  starttime =. 6!:9 ''
  v :: {{
    errNo =. dberr''
    errMsg =. 'The following error was encountered:',LF,dberm''
    Display^:(errNo~:12) 'FAILED';errMsg
  }} ''
  endtime =. 6!:9 ''
  elapsed =. (endtime - starttime) % 6!:8 ''
  Display 'COMPLETEDIN' ; 0j6 ": elapsed
}}

Doesn't this output multiple <PASSED::>? By using assert, I was thinking of making It to output <PASSED::> if nothing was thrown in it (Expect shouldn't have Display 'PASSED'). That's how most test frameworks work.

If it's not possible to write Describe 'feature' {{ and It 'behavior' {{, then Describe and It should probably be renamed. It's from BDD style test frameworks to write specs reading like describe "a feature"; it "should do this". Reading 'a feature' Describe; 'should do this' It is awkward. Not sure what's good, though. TestGroup and Test? It can also be some symbols is it's allowed and if there's anything suitable.

kazk commented 1 year ago

I forgot about this, but I think we're almost there.

@Bubbler-4 Can you create a repository similar to https://github.com/codewars/ocaml? Create it under your account, and we can transfer when it's ready.

If not, is anyone else interested?

The repo should include:

Dockerfile: Install J, create user, create /workspace
bin/run: A Bash script with commands to run an example kata in a container.
examples/*: Some example kata with passing tests, failing tests, errors, etc.
Running the tests should produce outputs described in CodeRunner Messages.

You can test how the output is rendered on Codewars with Kumite.

Open https://www.codewars.com/kumite/new?language=python
Disable tests

Print the output

print("""

<PASTE THE OUTPUT HERE>

""")

Run

Please let me know if you have any questions or need help.

codewars / runner

Add J #20

version

codemirror

icon