CI for forward-porting GC3 patches to GC4

[ddeclerck]

Follow-up of #146.

[GitMensch]

oops, hope I haven't broken the gitignore in f36dcda83d0bbfd6b03410e566680384b54dae43 - if not then we likely should apply that to the gcos3x branch as well.

[ddeclerck]

I suggest to wrap the commits again. From what I've inspected we need one refactor for integrating 4.x logic (you've spotted that well) nicely.

Saw your message a bit late, added another commit in the meantime 😅

By wrapping up you mean, committing to SVN ? (after doing the requested modifications of course)

[GitMensch]

This approach is reasonable in general.

... So you did already check that this piece of code changed in a later commit?

If you see a difference in the current code, then you can use "svn annotate" to check which commit did the change in this party of fileio.c to know what to merge before the refactoring.

:-)

Am 1. Juni 2024 00:07:13 MESZ schrieb OCP David Declerck @.***>:

@ddeclerck commented on this pull request.

• / apply COB_FILE_PATH if set (similar to ACUCOBOL's FILE-PREFIX) /
• if (file_paths) {
• for(k=0; file_paths[k] != NULL; k++) {
• snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s",
• file_paths[k], SLASH_CHAR, file_open_name);
• file_open_buff[COB_FILE_MAX] = 0;
• if (access (file_open_buff, F_OK) == 0) {
• break;
• } +#if defined(WITH_CISAM) || defined(WITH_DISAM) || defined(WITH_VBISAM) || defined(WITH_VISAM)
• / ISAM may append '.dat' to file name /
• snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s.dat",
• file_paths[k], SLASH_CHAR, file_open_name);
• file_open_buff[COB_FILE_MAX] = 0;
• if (access (file_open_buff, F_OK) == 0) {
• snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s",
• file_paths[k], SLASH_CHAR, file_open_name);
• file_open_buff[COB_FILE_MAX] = 0;
• break;
• } +#endif
• }
• if (file_paths[k] == NULL) {
• snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s",
• file_paths[0], SLASH_CHAR, file_open_name);
• file_open_buff[COB_FILE_MAX] = 0;
• }
• strncpy (file_open_name, file_open_buff, (size_t)COB_FILE_MAX);
• }

I remember why I haven't refactored that straight away : in case subsequent commits modify the same code, conflicts will be much easier to handle. I was thinking about keeping the refactoring for after all the patches are merged...

-- Reply to this email directly or view it on GitHub: https://github.com/OCamlPro/gnucobol/pull/147#discussion_r1622987080 You are receiving this because you commented.

Message ID: @.***>

[ddeclerck]

I tend to be overly "conservative". Indeed this piece of code is barely modified afterwards, so I'll do the refactoring.

[ddeclerck]

Is this okay to merge (@GitMensch) ?

[GitMensch]

I'll try to review this (late) evening.

[GitMensch]

looks_absolute should use "src", not file_open_name directly (merge issue?)

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

[ddeclerck]

looks_absolute should use "src", not file_open_name directly (merge issue?)

This change is introduced in a later commit (3993).

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

By "new use", de you mean the fact that we apply file paths to the complex case ?

[GitMensch]

By "new use", de you mean the fact that we apply file paths to the complex case ?

yes

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

Depends on how other functions do it - it is best for now to mimic that (once the merge is completed we may revisit that part, but there are "some" commits left until we get there).

[ddeclerck]

I made the necessary changes.

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

I find it more convenient to merge consecutive commits. If that's okay for you I could add to the current batch the next eligible commits until 3993 (that would be 6 commits: 3973, 3979, 3988, 3989, 3992 and 3993).

[GitMensch]

That batch is good to go :-)

[ddeclerck]

That batch is good to go :-)

Merged in SVN ;)

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

[GitMensch]

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

No, only new files are copied, the others left as-is; before a release I regenerate the files but the files are nearly completely maintained by the translation project. So just record the merge, then copy over new files and svn add them, if there are any.

And of course

[ddeclerck]

Alright.

And of course

Hope this unfinished sentence did not have any vital info 😅

[GitMensch]

I can't remember any important info missing there.

[codecov-commenter]

:warning: Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.

Please upload report for BASE (gc4@8117773). Learn more about missing BASE report.

Files	Patch %	Lines
cobc/pplex.l	83.33%	1 Missing :warning:
cobc/typeck.c	0.00%	0 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## gc4 #147 +/- ## ====================================== Coverage ? 63.71% ====================================== Files ? 39 Lines ? 64719 Branches ? 17969 ====================================== Hits ? 41234 Misses ? 16365 Partials ? 7120 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[ddeclerck]

Quick question: I sometimes see alternative code for GC4 in #if 0 blocks; I guess I should implement those and drop the other branch, correct ?

I'm talking about those:

#if 0 /* TODO for 4.0: set the attributes from the field given outside on the stack */
        output ("cob_field *cob_fret, const int cob_pam");
#else
        output ("cob_field **cob_fret, const int cob_pam");
#endif

[GitMensch]

That's quite a bunch - any reason to not merge upstream? Open issues you are aware of or special adjustments needed?

[we really need to get to commits that have someone else in the ChangeLogs...]

[ddeclerck]

That's quite a bunch - any reason to not merge upstream? Open issues you are aware of or special adjustments needed?

No good reason. It may be many commits, but the first batch had way more lines (this one is only +1,162 −904). Anyways, I'll merge upstream if that's okay with you.

[we really need to get to commits that have someone else in the ChangeLogs...]

I'm looking forward to reaching commit 4614 - our first contribution to GC3 ;)

[ddeclerck]

Ready to merge this batch - just awaiting confirmation ;)

[GitMensch]

confirmed

[ddeclerck]

In the process of merging commit 4332 from GC3 to GC4. It introduces many new exception definitions in exception.def, in particular COB_EC_CONTINUE (1800). However a (non-standard ?) exception EC-DELETE-FILE with code 1800 was also introduced in GC4 by commit 3645.

Are those numbers somehow standardized ? Can I just give a different code to EC-DELETE-FILE ?

[GitMensch]

The numbers are only internal, feel free to adjust new ones during merge as you see fit. For the delete file exception - this was part of a working draft for the standard and was later changed - if you could try to fix that as a new commit targetting 3.x (and may be merged together with these the referenced merge commit or in a later one) that would be very good.

Working draft: there is this exception to be able to recognize (I think the case of the file not being there and "success" NOT ON EXCEPTION be the result).

Standard (that's much better):

• the matching IO status is set; so for example 37/39 for files with missing access rights, different file attributes; 05 for "file was not there"
• the EC-I-O exception matching the io status is set

I think I've missed that later change in the standard and therefore had not adjusted 3.x. Can you please check its current state here? Could you please send a PR to fix what's needed to be fixed?

[GitMensch]

I suggest getting the commit of #152 also into gc4 branch if it isn't in yet (therefore it will also be used for the PR checks), then get this current bunch into upstream again.... but don't feel any pressure for each of those, you can add more merge-commits as you see fit and also skip the CI adjustment.

[ddeclerck]

I suggest getting the commit of #152 also into gc4 branch if it isn't in yet (therefore it will also be used for the PR checks), then get this current bunch into upstream again.... but don't feel any pressure for each of those, you can add more merge-commits as you see fit and also skip the CI adjustment.

Actually, the contents of #152 is already present both in the gcos4gnucobol-3.x and gc4 branches.

Indeed it's time to merge upstream again, as the next commit might be a bit tricky (EXTFH changes, though it's a merge commit from trunk, so it may in fact just be trivial).

[ddeclerck]

I think that is enough for this batch ;).

[GitMensch]

That batch looks good for me as well. I guess the EXTFH merge commit was not included yet - or was it just trivial?

[ddeclerck]

That batch looks good for me as well. I guess the EXTFH merge commit was not included yet - or was it just trivial?

Well, it was trivial to merge, as it was a merge commit from trunk to gnucobol-3.x. Though it did take some time to check that everything from this commit was "still there" in trunk, considering that things changed in trunk in the meantime.

[ddeclerck]

And here is another batch. Not too big this time. Next batch will (finally) include some of our contributions ;)

[ddeclerck]

Well, I guess that's it for this (quite big) batch. Is that okay for you @GitMensch ?

[GitMensch]

As noted I'm not sure about the complete merge for the stacktrace/dump and arguments, but otherwise I'd be fine with the current batch as-is.

[GitMensch]

That leaves us with merge the bunch as is before the minor GC3 change to tests/* for changelog and msvc (and maybe cobc/cobc.c) which will be merged when not conflicting to GC4.

[ddeclerck]

That leaves us with merge the bunch as is before the minor GC3 change to tests/* for changelog and msvc (and maybe cobc/cobc.c) which will be merged when not conflicting to GC4.

All done (except merging the minor changes to GC4 - that will hapen when it will happen :p). I also rebased my GC3 MSVC PR.

[ddeclerck]

Hmm, the introduction of the "generic registers" feature is getting tricky. I had to make tweaks (see the extra commit), but I'm not satisfied with the result :

• had to exclude registers (regardless of the count) in listings to get the same output as GC3 (this might be because cb_validate_field systematically increments the count in GC4, while it does not in GC3, but not only - I noticed the GC4 symbol table feature also increments by one the count of every field and register)
• had to exclude registers from the symbol table, as it would generate incorrect code (note that the GC4 symbol table prior to the inclusion of this feature does not include the registers either, so it may be just fine here)
• had to tweak the expected dump output in run_misc.at as it would not include the RETURN-CODE register no matter what (note that prior to this feature, the dump did not include this register in GC4, though the patch from GC3 does add this register...)

Though we may consider those to be design choices. Doesn't have to be identical to GC3, but we should clearly decide what we want in each kind of output.

[ddeclerck]

Less tweaks this time, but still had to adjust the dump output in run_misc.at. I think I'll just move on and put myself a reminder.

[GitMensch]

Wouldn't it be possible to postpone merging the register part to a later set?

[ddeclerck]

Wouldn't it be possible to postpone merging the register part to a later set?

Guess I'll just do that, skipping 4666-4668 for now.

[GitMensch]

Sounds good, skipping r4666-r4669 should give you a bunch of changes that can be merged (just ensure your mergeinfo is correct before checking in upstream); if you later get a conflict because of missing pieces either #if 0 /* TODO include after mergin r466X */ the related parts out or copy = "manual merge" the necessary parts over, moving the conflict-solving to a later time.

Note: I could have a look at the bunch in about two hours,; if you want to we could include the then-working part upstream before you have more time for the next one.

[ddeclerck]

Note: I could have a look at the bunch in about two hours,; if you want to we could include the then-working part upstream before you have more time for the next one.

I'm afraid there won't be much in this batch this time, I'm struggling with one of our own commits (PICTURE L), behaving differently because of differences in handling ODO / ODO slide.

[GitMensch]

Take the time and consider the next bunch to be merged in 10 days. If you feel stuck during that time feel free to drop me a ping on Matrix. Otherwise: see you then, happy hacking!

[ddeclerck]

Take the time and consider the next bunch to be merged in 10 days. If you feel stuck during that time feel free to drop me a ping on Matrix. Otherwise: see you then, happy hacking!

I think I'll let you enjoy your vacation ;)

[ddeclerck]

That's not many commits, but the number of lines is quite big already - I'd say this should be merged.

[GitMensch]

I'll have a look in the next two days.

[ddeclerck]

please check that single padding issue - and have a look at the MSVC testcases as there are 142 unexpected failures

I'm aware of these MSVC failures, actually a lot of them are trigger either because of missing gcdiff (does not seem to be built under MSVC), or with a "module libcobdb-1.dll not found" error (don't even know why this would happen...)

[GitMensch]

I'm aware of these MSVC failures

Hm, so all of the current failures are unrelated to this PR?

actually a lot of them are trigger either because of missing gcdiff (does not seem to be built under MSVC)

We should either drop gcdiff completely or use it more in the testsuite and build it with MSVC and OrangeC as well (I hope we do build the file tool, do we?)

or with a "module libcobdb-1.dll not found" error (don't even know why this would happen...)

Are the project files all adjusted to build the sub-files for delay load? Is the delay-load coded correctly for MSVC?

[ddeclerck]

I'm aware of these MSVC failures

Hm, so all of the current failures are unrelated to this PR?

Yeah, they were there before. I just check each time that I do not increase the number of failed tests under MSVC.

actually a lot of them are trigger either because of missing gcdiff (does not seem to be built under MSVC)

We should either drop gcdiff completely or use it more in the testsuite and build it with MSVC and OrangeC as well (I hope we do build the file tool, do we?)

gcdiff looks like a good idea ; seems it was introduced in the GC4 branch long ago but never made it to GC3. No opinion as to whether we should keep it or not though.

or with a "module libcobdb-1.dll not found" error (don't even know why this would happen...)

Are the project files all adjusted to build the sub-files for delay load? Is the delay-load coded correctly for MSVC?

I couldn't tell. Seems the MSVC build didn't get much love in the GC4 branch 😅 (I'm not a big than either...). I was thinking about fixing those MSVC builds once all patches are merged (I believe it will take quite some time).

[GitMensch]

I'd prefer first fixing the MSVC issues before going on with the PRs - but just checking that no new ones get in is reasonable as well. I'm fine with that.

Concerning gcdiff: this was added by Ron years ago to remove invocations of SED, especially for the listing header in the related tests, also because the sed script used back then wasn't portable (I think the current one is). Can you be so bold to send a mail to the dev list with the question what to do about gcdiff:

• drop that (means minor adjustment to the Makefile.am, deletion of the source file(s), and adjustments to the testsuite [I guess most can already be copied from 3.x)
• commit to use it:
  • get rid of all sed invocations in the testsuite if possible - add a new option to gcdiff for filtering other things
  • change gcdiff to open the files in binary mode (at least by default, so we don't ignore line break differences when we don't want to)
  • decide if gcdiff should be installed in user-space or not and adjust Makefile.am / package scripts as needed; if it is installed then it should also have a manpage (generated from --help is fine ) and its (sub)chapter in gnucobol.texi [both of this is definitely necessary for GC4 and cobfile - please come up with a PR for that if possible]
  • add it for MSVC+OrangeC builds [definitely necessary for cobfile as well]

You may have more to say to that and/or note your opinions/preferences as well. I tend to drop it, but that's likely because of an assumption that we may put time into something "unrelated to the mission" and would work contrary to the "one tool to do one job" principle [creating a substitute for Posix and GNU standard tools diff + sed with only partial functions of those]

We should have a dev decision on that and then also follow one of those two paths outlined above (as a separate commit from the merge commits, ideally "soon").

For the current bunch of commits - apart of the new #if 0 padding, I think the bunch is fine.

[ddeclerck]

or with a "module libcobdb-1.dll not found" error (don't even know why this would happen...)

Are the project files all adjusted to build the sub-files for delay load? Is the delay-load coded correctly for MSVC?

After a quick check, it seems the MSVC projets do not seem to be aware of this new feature. All the backend's C files are listed in the project files though (as part of libcob).

[ddeclerck]

Can you be so bold to send a mail to the dev list with the question what to do about gcdiff:

Do you think I am legitimate enough to send this message ? 😅

In any case, I don't think I have access to that mailing list (yet ?).

[ddeclerck]

For the current bunch of commits - apart of the new #if 0 padding, I think the bunch is fine.

Fixed, going to merge this one and start a new batch.