search

OCamlPro / gnucobol

A clone of the sourceforge GnuCOBOL compiler from COBOL to C.
https://get-superbol.com
GNU Lesser General Public License v3.0
16 stars 21 forks source link

[WIP] Add .github and .gitignore for better Github support #76

Closed lefessan closed 2 months ago

lefessan commented 1 year ago

The main issue with this PR is that the target is wrong. It should only target OCamlPro:gcos4gnucobol-3.x and is not applicable to be merged upstream.

I don't understand what you mean. If it finally works, or maybe even only for Ubuntu, why not accept it upstream if it makes the life easier for git and github users ?

GitMensch commented 1 year ago

The CI definition works in the gcos4gnucobol branch where we target all merges. The CI we currently use is appveyor - and those definitions are also not checked-in (yet). Otherwise I'm using Gitpod - also not checked in upstream (and others use some amazon and gitlab and ... integration).

lefessan commented 1 year ago

Oh, I see, one problem comes from the fact that I changed the default branch to gnucobol-3.x, because I thought it gave a bad idea of the repo, especially as gcos4gnucobol-3.x had not been updated for a long while (David told me this morning he wants to update it one 3.x), and gnucobol-3.x had today a better GCOS support than that branch.

Anyway, it would still be great to have a .gitignore file at toplevel.

lefessan commented 1 year ago

I have just put back gcos4gnucobol-3.x as the default branch.

lefessan commented 1 year ago

At least, Macos workflow is working, let's hope for Windows soon...

GitMensch commented 1 year ago

Looks like the use gcos4gnucobol-3.x branch is quite outdated. I suggest to:

codecov-commenter commented 1 year ago

Codecov Report

Merging #76 (c2c3fa5) into gcos4gnucobol-3.x (516b813) will not change coverage. The diff coverage is n/a.

@@                Coverage Diff                 @@
##           gcos4gnucobol-3.x      #76   +/-   ##
==================================================
  Coverage              65.27%   65.27%           
==================================================
  Files                     31       31           
  Lines                  56425    56425           
  Branches               14723    14723           
==================================================
  Hits                   36830    36830           
  Misses                 13807    13807           
  Partials                5788     5788

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

lefessan commented 1 year ago

By the way, do you have an idea of why the tests are failing on Windows ? For example, here: https://github.com/OCamlPro/gnucobol/actions/runs/3968237465/jobs/6801141707

lefessan commented 1 year ago

Do you have an example of a working appVeyor script for Windows ?

GitMensch commented 1 year ago

Note: "Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/checkout@v2. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/." the current version of actions/checkout seems to be version 3.3, maybe it is fine to drop the version (or use a current one)?

GitMensch commented 1 year ago

By the way, do you have an idea of why the tests are failing on Windows?

Let's have a look at testsuite.log "Detailed failed tests":

4. used_binaries.at:219: testing compiler outputs (file specified) ...

../../tests/used_binaries.at:269: $COBC -I sub/copy prog.cob -ext=cpy -o prog.i -MF prog.d -MT "prog.c prog.h" -MT prog$COB_EXE_EXT -MT prog.$COB_OBJECT_EXT -MT prog.i  -fsyntax-only
../../tests/used_binaries.at:270: $GREP 'prog.c prog.h '        prog.d
stdout:
prog.c prog.h prog.exe prog.o prog.i: \
../../tests/used_binaries.at:271: $GREP ' prog.i:'        prog.d
stdout:
prog.c prog.h prog.exe prog.o prog.i: \
../../tests/used_binaries.at:272: $GREP 'sub/copy/PROC.cpy'        prog.d
stdout:
../../tests/used_binaries.at:272: exit code was 1, expected 0
4. used_binaries.at:219: 4. compiler outputs (file specified) (used_binaries.at:219): FAILED (used_binaries.at:272)

This very likely fails because of the slashes. The testsuite misses the following lines, which I'll add in the next commit, if they were in we could have an easy look.

AT_CAPTURE_FILE([prog.i])
AT_CAPTURE_FILE([prog.c])
AT_CAPTURE_FILE([prog.d])

As not all grep implementations support regex the same way we could add another check for the backslash.

Can you test the following, please?

AT_CAPTURE_FILE([prog.i])
AT_CAPTURE_FILE([prog.c])

AT_CHECK([$COBC -I sub/copy prog.cob -o prog.c], [0], [], [])
AT_CHECK([$COBC -I sub/copy prog.c -o prog.$COB_OBJECT_EXT], [0], [], [])
AT_CHECK([$COBC -I sub/copy prog.$COB_OBJECT_EXT -o prog.$COB_MODULE_EXT])
AT_CHECK([$COBCRUN prog], [0], [bluBb END], [])
AT_CHECK([$COBC -I sub/copy -x prog.cob -o prog.c], [0], [], [])
AT_CHECK([$COBC -I sub/copy -x prog.c -o prog.$COB_OBJECT_EXT], [0], [], [])
AT_CHECK([$COBC -I sub/copy -x prog.$COB_OBJECT_EXT -o progo$COB_EXE_EXT], [0], [], [])
AT_CHECK([$COBCRUN_DIRECT ./progo], [0], [bluBb END], [])
# making the extension explicit here to not let case-insensitive file-systems catch a .CPY...
AT_CHECK([$COBC -I sub/copy prog.cob -ext=cpy -o prog.i], [0], [], [])
AT_CHECK([$COBC -x prog.i -o prog$COB_EXE_EXT], [0], [], [])
AT_CHECK([$COBCRUN_DIRECT ./prog$COB_EXE_EXT], [0], [bluBb END], [])
AT_CHECK([$COBC -x prog.i -fgen-c-line-directives -fgen-c-labels -save-temps], [0], [], [])
AT_CHECK([$GREP 'prog.i'   prog.c], [0], ignore, [])
AT_CHECK([$GREP 'prog.i'   prog.c | $GREP '#line'], [1], ignore, ignore)
AT_CHECK([$GREP 'prog.cob' prog.c | $GREP '#line'], [0], ignore, [])
AT_CHECK([$GREP 'PROC.cpy' prog.c | $GREP '#line'], [0], ignore, [])
AT_CHECK([$GREP 'ENTRY_PROG:'          prog.c], [0], ignore, [])
AT_CHECK([$GREP 'SECTION_END__PROC:'   prog.c], [0], ignore, [])
AT_CHECK([$GREP 'PARAGRAPH_00_l_4:'    prog.c], [0], ignore, [])
AT_CHECK([$GREP 'PARAGRAPH_EX_l_7:'    prog.c], [0], ignore, [])
AT_CHECK([$COBCRUN_DIRECT ./prog], [0], [bluBb END], [])

AT_CAPTURE_FILE([prog.d])
AT_CHECK([$COBC -I sub/copy prog.cob -ext=cpy -o prog.i -MF prog.d -MT "prog.c prog.h" -MT prog$COB_EXE_EXT -MT prog.$COB_OBJECT_EXT -MT prog.i  -fsyntax-only], [0], [], [])
AT_CHECK([$GREP 'prog.c prog.h '       prog.d], [0], ignore, [])
AT_CHECK([$GREP ' prog.i:'             prog.d], [0], ignore, [])
AT_CHECK([$GREP 'sub/copy/PROC.cpy'    prog.d], [0], ignore, [],
  # Previous test "failed" --> no entry with slash available, check backslash
  AT_CHECK([$GREP 'sub\copy\PROC.cpy'  prog.d], [0], ignore, []))
AT_CHECK([$GREP 'sub/PROCE.cpy'        prog.d], [0], ignore, [])
AT_CHECK([$GREP 'sub/PROCE.cpy'        prog.d], [0], ignore, [],
  # Previous test "failed" --> no entry with slash available, check backslash
  AT_CHECK([$GREP 'sub\PROCE'          prog.d], [0], ignore, []))

Next one:

8. used_binaries.at:352: testing temporary path invalid ...
../../tests/used_binaries.at:368: TMPDIR="" TMP="notthere" TEMP="" $COMPILE prog.cob
../../tests/used_binaries.at:371: $COBCRUN_DIRECT ./prog
../../tests/used_binaries.at:372: TMPDIR="" TMP="" TEMP="./prog.cob" $COMPILE prog.cob
--- -   2023-01-20 13:59:25.754442200 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/8/stderr    2023-01-20 13:59:25.703818600 +0000
@@ -1,2 +1,6 @@
 libcob: warning: Temporary directory TEMP is invalid, adjust TMPDIR!
+Cannot create temporary file in ?: Invalid argument
+gcc: internal compiler error: Aborted signal terminated program collect2
+Please submit a full bug report, with preprocessed source (by using -freport-bug).
+See <https://github.com/msys2/MINGW-packages/issues> for instructions.

So: please send in a bug report there and link it here :-)

What is likely happening: the invalid "TEMP" is catched by libcob and then unset, but possibly the GCC still gets the "old" environment and then stumbles over the invalid "TEMP". You possibly could test that with compiling any C program with this GCC and setting TEMP (which should be a directory) to this C program.

735. run_misc.at:4263: testing direct CALL in from C w/wo error; no exit ...
../../tests/run_misc.at:4374: $COMPILE caller.c
../../tests/run_misc.at:4375: $COMPILE_MODULE callee.cob callee2.cob buggy.cob
../../tests/run_misc.at:4376: $COBCRUN_DIRECT ./caller callee 00
../../tests/run_misc.at:4379: $COBCRUN_DIRECT ./caller callee 42
../../tests/run_misc.at:4382: $COBCRUN_DIRECT ./caller callee2
--- -   2023-01-20 14:02:36.583944900 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/735/stderr  2023-01-20 14:02:36.524776600 +0000
@@ -1,2 +1 @@
-note: STOP RUN with return code 2

--- -   2023-01-20 14:02:36.636258200 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/735/stdout  2023-01-20 14:02:36.587278100 +0000
@@ -1 +1 @@
-STOP WITH 2
+
../../tests/run_misc.at:4382: exit code was 127, expected 0
735. run_misc.at:4263: 735. direct CALL in from C w/wo error; no exit (run_misc.at:4263): FAILED (run_misc.at:4382)

That should be debugged by someone.

761. run_misc.at:7140: testing CALL RETURNING POINTER ...
../../tests/run_misc.at:7189: $COMPILE -fno-gen-c-decl-static-call prog.cob
../../tests/run_misc.at:7190: $COMPILE progdyn.cob
../../tests/run_misc.at:7191: $COBCRUN_DIRECT ./prog
../../tests/run_misc.at:7192: $COBCRUN_DIRECT ./progdyn
--- /dev/null   2023-01-20 14:02:45.000000000 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/761/stderr  2023-01-20 14:02:45.640017100 +0000
@@ -0,0 +1 @@
+libcob: progdyn.cob:12: error: module 'cob_getenv' not found
../../tests/run_misc.at:7192: exit code was 1, expected 0

Hm the function is not found, maybe the test needs an adjustment, maybe the function lookup on Win32.

773. run_misc.at:10227: testing stack and dump feature ...
../../tests/run_misc.at:10531: $COMPILE prog.cob sub2.cob
../../tests/run_misc.at:10536: COB_STACKTRACE=1 COB_DUMP_FILE=tstdump.dump \
$COBCRUN_DIRECT ./prog
--- -   2023-01-20 14:02:50.092696900 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/773/stderr  2023-01-20 14:02:50.011768400 +0000
@@ -12,5 +12,5 @@
    CALL-IT-OMIT at prog.cob:118
    MAIN-100 at prog.cob:85
    ENTRY prog at prog.cob:77
- Started by ./prog
+ Started by D:\a\gnucobol\gnucobol\_build\tests\testsuite.dir\0773\prog.exe

Apart from showing that there's a "gnucobol" directory "too much" in the CI: MSYS does a path translation here executing it with a full path. This can possibly be adjusted by some environment variable. An alternative would be using cobcrun instead, but that's a different test and also done. For now I've switched the cobcrun based tests up-front (for the next commit), but that will fail again at this place. Either we find an environment variable to adjust this, or we hack that ourself by an option to only show the executable without the path in the dump output.

The same applies to test 774

+<  Started by ./prog
+---
+>  Started by D:\a\gnucobol\gnucobol\_build\tests\testsuite.dir\0774\prog.exe

The last two failures seem to be "real failures":


#                             -*- compilation -*-
898. run_file.at:11845: testing LINE SEQUENTIAL REWRITE ...
../../tests/run_file.at:12039: $COMPILE prog.cob
../../tests/run_file.at:12041: $COBCRUN_DIRECT ./prog
--- -   2023-01-20 14:03:37.087528200 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/898/stdout  2023-01-20 14:03:37.047755400 +0000
@@ -10,25 +10,11 @@
 Read  ALP00000 Trms:08240
 Read  BET00000 Trms:00010
 REWRITE BET00000 Sts 00 Trms:00011
-Read  GAM00000 Trms:00013
+Read  319      Trms:     
 Read EPS00000 Sts:00
 REWRITE EPS00000 Sts 00 Trms:08240
 List back Open Sts:00
-Read  ALP00000 Trms:08240
-Read  BET00000 Trms:00011
-Read  GAM00000 Trms:00013
-Read  DEL00000 Trms:65535
-Read  EPS00000 Trms:08240
-Read  FOR00000 Trms:00254
-Read Status: 10
+Read Status: 06
 List after extend Open Sts:00
-Read  ALP00000 Trms:08240
-Read  BET00000 Trms:00011
-Read  GAM00000 Trms:00013
-Read  DEL00000 Trms:65535
-Read  EPS00000 Trms:08240
-Read  FOR00000 Trms:00254
-Read  ALP00000 Trms:08240
-Read  BET00000 Trms:00010
-Read Status: 10
+Read Status: 06

898. run_file.at:11845: 898. LINE SEQUENTIAL REWRITE (run_file.at:11845): FAILED (run_file.at:12041)

#                             -*- compilation -*-
900. run_file.at:12274: testing Concatenated Files ...
../../tests/run_file.at:12392: $COMPILE prog.cob
../../tests/run_file.at:12394: $COBCRUN_DIRECT ./prog
../../tests/run_file.at:12399: COB_SEQ_CONCAT_NAME=true $COBCRUN_DIRECT ./prog
--- -   2023-01-20 14:03:37.723703200 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/900/stdout  2023-01-20 14:03:37.672984800 +0000
@@ -1,10 +1,5 @@
 file1         Record 1  .
 file1         Record 2  .
 file1         Record 3  .
-file2         Record 1  .
-file2   Hello Record 2  .
-file2         Record 3  .
-file3         Record 1  .
-file3         Record 2  .
-file3         Record 3  .
+Read Status: 06

900. run_file.at:12274: 900. Concatenated Files (run_file.at:12274): FAILED (run_file.at:12399)

Someone with a similar environment should debug this and try to fix fileio.c.

GitMensch commented 1 year ago

Do you have an example of a working appVeyor script for Windows ?

Yes. Note that there's no msys64 in there yet, I'm open for a "patch" to that configuration (it is completely configured via UI), here are the exported definitions and a link to the build history appveyor-win32-posix.yml - build history appveyor-vs.yml - build history

lefessan commented 1 year ago

I think I am going to split this PR in 3:

GitMensch commented 1 year ago
  • a working workflow for Macos, and Windows without tests

That would be useless. If we can't get the Windows one to pass, then it is still better to run the tests (with the nproc stuff)

- make check || ("echo not all tests passed" && false)
- make test

so we still can investigate the logs.

GitMensch commented 1 year ago

Note for MSYS builds: there are variables MSYS_NO_PATHCONV and MSYS2_ARG_CONV_EXCL and likely more. I'm not sure if someone has checked those already. As noted: for widows it would be reasonable to ignore the result of make check (still doing that and uploading the testsuite.log) and only fail of make test fails.

GitMensch commented 1 year ago

The last two failures seem to be "real failures":


#                             -*- compilation -*-
898. run_file.at:11845: testing LINE SEQUENTIAL REWRITE ...
../../tests/run_file.at:12039: $COMPILE prog.cob
../../tests/run_file.at:12041: $COBCRUN_DIRECT ./prog
--- - 2023-01-20 14:03:37.087528200 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/898/stdout    2023-01-20 14:03:37.047755400 +0000
@@ -10,25 +10,11 @@
 Read  ALP00000 Trms:08240
 Read  BET00000 Trms:00010
 REWRITE BET00000 Sts 00 Trms:00011
-Read  GAM00000 Trms:00013
+Read  319      Trms:     
 Read EPS00000 Sts:00
 REWRITE EPS00000 Sts 00 Trms:08240
 List back Open Sts:00
-Read  ALP00000 Trms:08240
-Read  BET00000 Trms:00011
-Read  GAM00000 Trms:00013
-Read  DEL00000 Trms:65535
-Read  EPS00000 Trms:08240
-Read  FOR00000 Trms:00254
-Read Status: 10
+Read Status: 06
 List after extend Open Sts:00
-Read  ALP00000 Trms:08240
-Read  BET00000 Trms:00011
-Read  GAM00000 Trms:00013
-Read  DEL00000 Trms:65535
-Read  EPS00000 Trms:08240
-Read  FOR00000 Trms:00254
-Read  ALP00000 Trms:08240
-Read  BET00000 Trms:00010
-Read Status: 10
+Read Status: 06

898. run_file.at:11845: 898. LINE SEQUENTIAL REWRITE (run_file.at:11845): FAILED (run_file.at:12041)

#                             -*- compilation -*-
900. run_file.at:12274: testing Concatenated Files ...
../../tests/run_file.at:12392: $COMPILE prog.cob
../../tests/run_file.at:12394: $COBCRUN_DIRECT ./prog
../../tests/run_file.at:12399: COB_SEQ_CONCAT_NAME=true $COBCRUN_DIRECT ./prog
--- - 2023-01-20 14:03:37.723703200 +0000
+++ /d/a/gnucobol/gnucobol/_build/tests/testsuite.dir/at-groups/900/stdout    2023-01-20 14:03:37.672984800 +0000
@@ -1,10 +1,5 @@
 file1         Record 1  .
 file1         Record 2  .
 file1         Record 3  .
-file2         Record 1  .
-file2   Hello Record 2  .
-file2         Record 3  .
-file3         Record 1  .
-file3         Record 2  .
-file3         Record 3  .
+Read Status: 06

900. run_file.at:12274: 900. Concatenated Files (run_file.at:12274): FAILED (run_file.at:12399)

Someone with a similar environment should debug this and try to fix fileio.c.

I've put more than 2 hours in testing, debugging and trying to get it working somehow. The later was not successful, but there's now an official bug item for that.

https://sourceforge.net/p/gnucobol/bugs/871

Note: some of the false positives should be fixed with one of the next commits.

GitMensch commented 2 months ago

closed as .github/.gitignore is currently not upstream and the change to the testsuite is already there