zaskar9 commented 4 months ago

This issue is intended to track the current state of the unit tests. Below is a summary of all tests that need work. They fall into the Unsupported, Failed, and Unexpectedly Passed categories. For every test, there is a brief summary of the problem or a description of the feature that is missing.

Unsupported

loop_1.mod: LOOP and EXIT statement missing → infinite loop
~~module_unterminated.mod: no terminating dot at end of module → infinite loop~~
procedure_missing_return.mod: no error on missing/unreachable RETURN

Failed

arithmetic_12.mod: data type BYTE missing
array_5.mod: data type PROCEDURE missing
case_1.mod: CASE statement missing
loop_2.mod: LOOP statement missing
system_6.mod: problem in Out.Real (NaN instead of INF/-INF)
~~system_7.mod: problem with signedness of 64-bit hexadecimal values~~
~~fail_on_extra_semicolon.mod: expected fail~~

Unexpectedly Passed

arithmetic_3.mod: should trigger integer overflow trap
arithmetic_5.mod: divisor cannot be negative (cf. O07.8.2.2)
arithmetic_6.mod: divisor cannot be negative (cf. O07.8.2.2)
arithmetic_9.mod: should trigger integer overflow trap
arithmetic_10.mod: should trigger division by zero trap
arithmetic_11.mod: Oberon has no implicit type conversions, should be disabled by default
arithmetic_15.mod: should trigger division by zero trap
array_6.mod: should trigger out-of-bounds trap
procedure_4.mod: Oberon-07 scoping rules not enforced: intermediate declarations cannot be accessed in nested procedures
~~if_with_trailing_end.mod: parser does not complain about superfluous END in IF-ELSIF-ELSE statement~~

Overall State

  Unsupported        :  2
  Passed             : 68
  Expectedly Failed  :  7
  Failed             :  5
  Unexpectedly Passed:  9

zaskar9 commented 4 months ago

Latest version of oberon-lang includes fixes to the scanner and parser to address the following issues.

~~module_unterminated.mod: no terminating dot at end of module → infinite loop~~
~~fail_on_extra_semicolon.mod: expected fail~~
~~if_with_trailing_end.mod: parser does not complain about superfluous END in IF-ELSIF-ELSE statement~~

tenko commented 3 months ago

Result MSYS2:

Failed Tests (7): Oberon :: codegen/arithmetic_12.mod Oberon :: codegen/array_5.mod Oberon :: codegen/case_1.mod Oberon :: codegen/const_long_hex.mod Oberon :: codegen/const_longint.mod Oberon :: codegen/loop_2.mod Oberon :: codegen/system_6.mod

Unexpectedly Passed Tests (7): Oberon :: codegen/arithmetic_11.mod Oberon :: codegen/arithmetic_15.mod Oberon :: codegen/arithmetic_3.mod Oberon :: codegen/arithmetic_5.mod Oberon :: codegen/arithmetic_6.mod Oberon :: codegen/arithmetic_9.mod Oberon :: codegen/procedure_4.mod

Unsupported : 2 Passed : 66 Expectedly Failed : 9 Failed : 7 Unexpectedly Passed: 7

codegen/const_long_hex.mod and codegen/const_longint.mod fails for me. I am guessing this is related to use of long which is 32bit on my platform.

zaskar9 commented 3 months ago

I just pushed two commits to the master branch that update Scanner, AST, Sema and CodeGen to use fixed-width integers (i.e., int64_t, int32_t, and int16_t for LONGINT, INTEGER, and SHORTINT as well as uint8_t for CHAR). Does that improve portability? It surely makes things more consistent.

tenko commented 3 months ago

codegen/const_long_hex.mod is now fixed.

codegen/const_longint.mod still failed.

I added the following to rt_out_int :

ifdef MINGW32

sprintf(buf, "%lld", i);

else

sprintf(buf, "%" PRId64, i);

endif

This fixes the problem. Probably the macro PRId64 is not correct on my platform.

zaskar9 commented 3 months ago

Since I currently don't have access to my Windows virtual machine, due to Easter travel, I was wondering if you could try the following. If you build the compiler in DEBUG mode and run it with the -v flag, it will print out a lot (like really a lot) of diagnostic information that will help me to localize the problem.

For example, if I run const_longint.mod in DEBUG it prints (among other things) the module how the compiler "sees" it. On my machine this look as shown below.

MODULE ConstLongInt(*Scope:0*);
IMPORT Out; 

   CONST longintmax(*Scope:1*) = 9223372036854775807(*L*)(*Type:LONGINT*);
         longintmin(*Scope:1*) = -9223372036854775808(*L*)(*Type:LONGINT*);

   PROCEDURE Test(*Scope:1*)();
   BEGIN
      Out.Long(9223372036854775807(*L*), 0(*S*));
      Out.Ln();
      Out.Long(-9223372036854775808(*L*), 0(*S*));
      Out.Ln()
   END Test;

BEGIN
   Test()
END ConstLongInt.

The first thing to make sure is that all constant literals are annotated by the correct type (S = SHORTINT, I = INTEGER, L = LONGINT, etc). If this is not the case, the problem most likely resides in the Scanner code assigns types to literals. If all literals are correctly typed, the problem will be in Out.Long().

As the compiler can do more and more, I am currently trying to implement more parts of Out and Texts in Oberon rather than in C. So, if the problem is in the output procedure, it hopefully resolves itself "automatically", but if it is in the Scanner, I will put it on the to-do list.

tenko commented 3 months ago

No problem. Output MSYS2:

MODULE ConstLongInt(Scope:0); IMPORT Out;

CONST longintmax(Scope:1) = 9223372036854775807(L)(Type:LONGINT); longintmin(Scope:1) = -9223372036854775808(L)(Type:LONGINT);

PROCEDURE Test(Scope:1)(); BEGIN Out.Long(9223372036854775807(L), 0(S)); Out.Ln(); Out.Long(-9223372036854775808(L), 0(S)); Out.Ln() END Test;

BEGIN Test() END ConstLongInt.

The Scanner is then OK.

Sidenote on VM images: The NativeOberon image is available at. This works suprisingly well and really fun to explore. Happy easter!

zaskar9 commented 3 months ago

On my Windows machine (VMware, Windows 11, x86_64), I have a strange problem that obfuscates a clear view of the status of the unit tests. The issue is related to some modules failing in JIT mode due to unresolved symbols. Using the MSYS/Clang64 toolchain the missing symbol is ___chkstk_ms, while using the MSVC toolchain the missing symbols are __security_cookie and __security_check_cookie. Both symbols are related to the stack protection mechanisms of the respective toolchains.

I checked the assembly generated by oberon-lang and calls to these symbols are indeed emitted if the optimization level is set to -O0, otherwise the calls are (probably) optimized away by a (yet to be identified) optimization pass. Since there is no option to run optimization in JIT mode, these errors cannot be avoided for certain modules for the time being. I have not yet fully understood under which circumstances these calls are emitted. The sets of modules affected by the problem are similar but not the same in the two toolchains.

I'm now trying to figure out if and how I can load the libraries into the JIT session that contain these symbols or whether it makes sense to run the optimization pass that removes these calls also in JIT mode.

tenko commented 3 months ago

I found this problem also and did not find any easy option to turn this off.

It seems that on un-optimized Windows build stack protection functions are added by default. These functions reside in the static library libclang_rt.builtins-x86_64.a as far as I can tell for MSYS2 & LLVM. For MSVC it is probably enough with the ucrtbase.dll (Only Windows 10 and later. msvcrt for earlier versions).

In order to solve this I always linked in this libraries on the Windows platform found in these commits 1 and 2. The latter commit I placed the static library alongside the .exe file so that upgrading CLANG MSYS2 will not break the installation.

An other option could be just to define some dummy functions covering this and add to JIT symbol table.

I understand from reading some LLVM mailing lists that the long-term solution being worked on is a separate runtime library for the JIT functionlality. This runtime library is still pending, so we might just choose to install a temporary solution as referenced above.

zaskar9 commented 3 months ago

Interesting! Do you have a link to the mailing list discussion?

tenko commented 3 months ago

I did not find the post, but I believe this issue is relevant : 60034

Edit : Turns out there is an option to link and export the symbols in the .exe directly: 1 2 This is maybe the cleanest solution as it will be resolved at compile time and can be just inserted with

ifdefs into the C++ code.

tenko commented 3 months ago

Confirm the fix for the stack protection issue is working for me. The result for unittests is:

Failed Tests (8): Oberon :: codegen/arithmetic_12.mod Oberon :: codegen/arithmetic_16.mod Oberon :: codegen/array_5.mod Oberon :: codegen/case_1.mod Oberon :: codegen/const_1.mod Oberon :: codegen/loop_2.mod Oberon :: codegen/system_6.mod Oberon :: codegen/system_7.mod

Unexpectedly Passed Tests (7): Oberon :: codegen/arithmetic_11.mod Oberon :: codegen/arithmetic_15.mod Oberon :: codegen/arithmetic_3.mod Oberon :: codegen/arithmetic_5.mod Oberon :: codegen/arithmetic_6.mod Oberon :: codegen/arithmetic_9.mod Oberon :: codegen/procedure_4.mod

Testing Time: 22.87s Unsupported : 2 Passed : 65 Expectedly Failed : 9 Failed : 8 Unexpectedly Passed: 7

codegen/system_7.mod fails with output -1.0E+00 against CHECK: -1.00E+00. This is probably due to update to Out module and similar on other platforms?

codegen/system_6.mod fails with NaN output where INF and -INF where expected for REAL.

zaskar9 commented 3 months ago

Interesting! These results are very different from what I see on the platforms available to me (including MSYS2/Clang64). Have you rebuilt liboberon before running the tests?

tenko commented 3 months ago

Sorry. Stale version of liboberon.

Result is:

Failed Tests (4): Oberon :: codegen/arithmetic_12.mod Oberon :: codegen/array_5.mod Oberon :: codegen/case_1.mod Oberon :: codegen/loop_2.mod

Unexpectedly Passed Tests (7): Oberon :: codegen/arithmetic_11.mod Oberon :: codegen/arithmetic_15.mod Oberon :: codegen/arithmetic_3.mod Oberon :: codegen/arithmetic_5.mod Oberon :: codegen/arithmetic_6.mod Oberon :: codegen/arithmetic_9.mod Oberon :: codegen/procedure_4.mod

Testing Time: 29.32s Unsupported : 2 Passed : 69 Expectedly Failed : 9 Failed : 4 Unexpectedly Passed: 7

zaskar9 / oberon-lang

Making Unit Tests Work! #28

ifdef MINGW32

else

endif

ifdefs into the C++ code.