Here's the testing scrips from BREXX/VM, converted and returning return codes on failed tests (instead of returning 0 and just printing out that it failed). The scripts tests.sh and test_one.sh in the test folder can be used for automation. Instructions are included in the test/README.md file. I also fixed broken tests (that were broken not bugs, i.e. C2X that were expecting ASCII, etc). Everything that fails now should be a bug.
One thing to note, scripts labelled not_working aren't broken, it appears that they use custom functions or other items not included with Brexx, i kept them in case people want to implement new features but they don't get included in the automation.
----------------------------------------
File abbrev.rexx
Look for ABBREV OK
ABBREV OK
----------------------------------------
File abs.rexx
Look for ABS OK
ABS OK
----------------------------------------
File address.rexx
Look for Address OK
[SYSTEM]
Address OK
----------------------------------------
File arg.rexx
Look for ARG OK
74 *-* if arg(999) \== ''
Error 40 running RXRUN, line 74: Incorrect call to routine
----------------------------------------
File b2x.rexx
Look for B2X OK
B2X OK
----------------------------------------
File bitand.rexx
Look for BITAND OK
BITAND OK
----------------------------------------
File bitor.rexx
Look for BITOR OK
BITOR OK
----------------------------------------
File bitxor.rexx
Look for BITXOR OK
BITXOR OK
----------------------------------------
File c2d.rexx
Look for C2D OK
failed in test 3
191 *-* if c2d('0031'x,0) \== 0
Error 40 running RXRUN, line 191: Incorrect call to routine
----------------------------------------
File c2x.rexx
Look for C2X OK
C2X OK
----------------------------------------
File center.rexx
Look for CENTER OK
CENTER OK
----------------------------------------
File changes.rexx
Look for CHANGESTR OK
CHANGESTR OK
----------------------------------------
File charin.rexx
Look for CHARIN OK
8 *-* call lineout file, "Line 1"
Error 57 running RXRUN, line 8: Cannot open file
----------------------------------------
File charout.rexx
Look for CHAROUT OK
8 *-* call charout file "F 6", "Line 1"cr
Error 57 running RXRUN, line 8: Cannot open file
----------------------------------------
File chars.rexx
Look for CHARS OK
7 *-* call lineout file "F 80", "Line 1"
Error 57 running RXRUN, line 7: Cannot open file
----------------------------------------
File compare.rexx
Look for COMPARE OK
COMPARE OK
----------------------------------------
File copies.rexx
Look for COPIES OK
COPIES OK
----------------------------------------
File countst.rexx
Look for COUNTSTR OK
COUNTSTR OK
----------------------------------------
File d2c.rexx
Look for D2C OK
D2C OK
----------------------------------------
File d2x.rexx
Look for D2X OK
D2X OK
----------------------------------------
File datatyp.rexx
Look for DATATYPE OK
failed in test 51
failed in test 52
failed in test 58
failed in test 59
DATATYPE OK
----------------------------------------
File delstr.rexx
Look for DELSTR OK
DELSTR OK
----------------------------------------
File delword.rexx
Look for DELWORD OK
DELWORD OK
----------------------------------------
File digits.rexx
Look for DIGITS OK
failed in test 1
DIGITS OK
----------------------------------------
File errorms.rexx
Look for ERRORTEXT OK
printing all errortext
2 [Failure during finalization]
3 [Failure during initialization]
4 [Program interrupted]
5 [System resources exhausted]
6 [Unmatched "/*" or quote]
7 [WHEN or OTHERWISE expected]
8 [Unexpected THEN or ELSE]
9 [Unexpected WHEN or OTHERWISE]
10 [Unexpected or unmatched END]
13 [Invalid character in program]
14 [Incomplete DO/SELECT/IF]
15 [Invalid hexadecimal or binary string]
16 [Label not found]
17 [Unexpected PROCEDURE]
18 [THEN expected]
19 [String or symbol expected]
20 [Name expected]
21 [Invalid data on end of clause]
22 [Invalid character string]
23 [Invalid data string]
24 [Invalid TRACE request]
25 [Invalid sub-keyword found]
26 [Invalid whole number]
27 [Invalid DO syntax]
28 [Invalid LEAVE or ITERATE]
29 [Environment name too long]
30 [Name or string too long]
31 [Name starts with number or "."]
33 [Invalid expression result]
34 [Logical value not "0" or "1"]
35 [Invalid expression]
36 [Unmatched "(" in expression]
37 [Unexpected "," or ")"]
38 [Invalid template or pattern]
40 [Incorrect call to routine]
41 [Bad arithmetic conversion]
42 [Arithmetic overflow/underflow]
43 [Routine not found]
44 [Function did not return data]
45 [No data specified on function RETURN]
46 [Invalid variable reference]
47 [Unexpected label]
48 [Failure in system service]
49 [Interpretation Error]
50 [Unrecognized reserved symbol]
51 [Invalid function name]
52 [Result returned by "<name>" is longer than #Limit_String 'characters]
53 [Invalid option]
54 [Invalid STEM value]
55 [Database Error]
56 [Shared library error "<dlopen>"]
57 [Cannot open file]
58 [File not found]
59 [File not opened]
60 [Memory allocation failed.]
61 [Memory reallocation failed.]
62 [Illegal DD name <value> ]
ERRORTEXT OK
----------------------------------------
File errorte.rexx
Look for ERRORTEXT OK
ERRORTEXT OK
----------------------------------------
File format.rexx
Look for FORMAT OK
failed in test 1
failed in test 2
failed in test 4
failed in test 5
failed in test 6
failed in test 7
failed in test 8
failed in test 9
failed in test 10
failed in test 11
failed in test 12
failed in test 13
failed in test 14
failed in test 16
failed in test 17
failed in test 20
failed in test 22
failed in test 23
failed in test 29
failed in test 32
failed in test 33
failed in test 34
failed in test 35
failed in test 36
failed in test 38
failed in test 39
failed in test 40
failed in test 41
failed in test 44
failed in test 45
failed in test 47
failed in test 49
failed in test 51
failed in test 52
failed in test 53
failed in test 54
failed in test 55
failed in test 56
failed in test 57
failed in test 58
failed in test 59
failed in test 60
failed in test 61
failed in test 62
failed in test 63
failed in test 64
failed in test 65
failed in test 66
failed in test 67
failed in test 69
failed in test 70
failed in test 71
failed in test 72
failed in test 73
failed in test 74
failed in test 75
failed in test 76
failed in test 77
failed in test 78
failed in test 79
failed in test 80
failed in test 81
failed in test 84
failed in test 85
failed in test 86
failed in test 89
failed in test 90
failed in test 91
failed in test 92
failed in test 93
failed in test 94
failed in test 95
failed in test 96
failed in test 97
failed in test 98
failed in test 99
failed in test 100
failed in test 101
failed in test 102
failed in test 103
failed in test 104
failed in test 105
failed in test 106
failed in test 107
failed in test 108
failed in test 109
failed in test 110
FORMAT OK
----------------------------------------
File form.rexx
Look for FORM OK
FORM OK
----------------------------------------
File fuzz.rexx
Look for FUZZ OK
FUZZ OK
----------------------------------------
File insert.rexx
Look for INSERT OK
----------------------------------------
File lastpos.rexx
Look for LASTPOS OK
failed in test 11
LASTPOS OK
----------------------------------------
File left.rexx
Look for LEFT OK
LEFT OK
----------------------------------------
File length.rexx
Look for LENGTH OK
LENGTH OK
----------------------------------------
File linein.rexx
Look for LINEIN OK
12 *-* call lineout file, "Line 1"
Error 57 running RXRUN, line 12: Cannot open file
----------------------------------------
File lineout.rexx
Look for LINEOUT OK
9 *-* call lineout file, "Line 1"
Error 57 running RXRUN, line 9: Cannot open file
----------------------------------------
File lines.rexx
Look for LINES OK
9 *-* call lineout file, "Line 1"
Error 57 running RXRUN, line 9: Cannot open file
----------------------------------------
File max.rexx
Look for MAX OK
MAX OK
----------------------------------------
File min.rexx
Look for MIN OK
MIN OK
----------------------------------------
File overlay.rexx
Look for OVERLAY OK
OVERLAY OK
----------------------------------------
File pos.rexx
Look for POS OK
failed in test 10
POS OK
----------------------------------------
File random.rexx
Look for RANDOM OK
Values may differ.
RANDOM()=549
RANDOM(5,8)=8
RANDOM(,,1989)=495
RANDOM(2)=389
RANDOM OK
----------------------------------------
File regress.rexx
Regression test with all sorts of pseudo-codes. Look for "Done OK"
failed test 25
196 *-* numeric fuzz 0
Error 26.6 running RXRUN, line 196: NUMERIC FUZZ value must be zero or a positive whole number; found "0"
----------------------------------------
File reverse.rexx
Look for REVERSE OK
REVERSE OK
----------------------------------------
File right.rexx
Look for RIGHT OK
RIGHT OK
----------------------------------------
File sign.rexx
Look for SIGN OK
SIGN OK
----------------------------------------
File space.rexx
Look for SPACE OK
SPACE OK
----------------------------------------
File strip.rexx
Look for STRIP OK
STRIP OK
----------------------------------------
File substr.rexx
Look for SUBSTR OK
SUBSTR OK
----------------------------------------
File subword.rexx
Look for SUBWORD OK
SUBWORD OK
----------------------------------------
File symbol.rexx
Look for SYMBOL OK
SYMBOL OK
----------------------------------------
File testdat.rexx
Look for DATE OK
30 Sep 2020
DATE OK
----------------------------------------
File time.rexx
Look for TIME OK
05:25:49
5:25am
5
05:25:48.81
325
05:25:49
19549
TIME OK
----------------------------------------
File trace.rexx
Look for TRACE OK
N
N
TRACE OK
----------------------------------------
File transla.rexx
Look for TRANSLATE OK
TRANSLATE OK
----------------------------------------
File trunc.rexx
Look for TRUNC OK
failed in test 4
failed in test 16
failed in test 19
failed in test 20
failed in test 22
failed in test 23
failed in test 24
failed in test 25
failed in test 26
failed in test 27
failed in test 30
failed in test 78
TRUNC OK
----------------------------------------
File value.rexx
Look for VALUE OK
VALUE OK
----------------------------------------
File verify.rexx
Look for VERIFY OK
VERIFY OK
----------------------------------------
File wordind.rexx
Look for WORDINDEX OK
WORDINDEX OK
----------------------------------------
File wordlen.rexx
Look for WORDLENGTH OK
WORDLENGTH OK
----------------------------------------
File wordpos.rexx
Look for WORDPOS OK
WORDPOS OK
----------------------------------------
File word.rexx
Look for WORD OK
WORD OK
----------------------------------------
File words.rexx
Look for WORDS OK
WORDS OK
----------------------------------------
File x2b.rexx
Look for X2B OK
X2B OK
----------------------------------------
File x2c.rexx
Look for X2C OK
X2C OK
----------------------------------------
File x2d.rexx
Look for X2D OK
failed in test 68
345 *-* if X2D('',0) \== '0'
Error 40 running RXRUN, line 345: Incorrect call to routine
----------------------------------------
File xrange.rexx
Look for XRANGE OK
XRANGE OK
And the output from ../build/rc.sh tests.jcl ../../tk4-test/prt/prt00e.txt:
Here's the testing scrips from BREXX/VM, converted and returning return codes on failed tests (instead of returning 0 and just printing out that it failed). The scripts
tests.sh
andtest_one.sh
in thetest
folder can be used for automation. Instructions are included in thetest/README.md
file. I also fixed broken tests (that were broken not bugs, i.e. C2X that were expecting ASCII, etc). Everything that fails now should be a bug.One thing to note, scripts labelled
not_working
aren't broken, it appears that they use custom functions or other items not included with Brexx, i kept them in case people want to implement new features but they don't get included in the automation.Example output at time of pull request:
...
And the output from
../build/rc.sh tests.jcl ../../tk4-test/prt/prt00e.txt
: