Enhancement: Create GitHub CI/CD workflow and/or client side commit hooks

[SirWumpus]

Each push to GitHub should run the Makefile test target (to ensure we only ever push working code) as part of a CI workflow.

Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

[xexyl]

Each push to GitHub should run the Makefile test target (to ensure we only ever push working code) as part of a CI workflow.

Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

Some tests (via workflow) are done but as for running make test I don't think it's even possible in full. And the real one that needs to be verified is make prep but that would require a lot.

On the other hand one can add a commit hook for their own that runs prior to committing.

... yes I brought this up before hence how I know about these things. Though of course that was some time ago so maybe things have changed.

I guess we'll have to wait to see what Landon thinks.

[lcn2]

Each push to GitHub should run the Makefile test target (to ensure we only ever push working code) as part of a CI workflow. Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

Some tests (via workflow) are done but as for running make test I don't think it's even possible in full. And the real one that needs to be verified is make prep but that would require a lot.

Agree.

On the other hand one can add a commit hook for their own that runs prior to committing.

That might be useful, @xexyl. Would that hook run make prep or make release or some other test?

[xexyl]

Each push to GitHub should run the Makefile test target (to ensure we only ever push working code) as part of a CI workflow.

Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

Some tests (via workflow) are done but as for running make test I don't think it's even possible in full. And the real one that needs to be verified is make prep but that would require a lot.

Agree.

On the other hand one can add a commit hook for their own that runs prior to committing.

That might be useful, @xexyl. Would that hook run make prep or make release or some other test?

Just as a reminder I did this once but you thought it might be the wrong time. Is it a good time now?

I can look and see if I still have what I did or else rewrite it.

BUT it might be possible to only have an example for people to use if they want to.

Let me know what you think.

[lcn2]

Each push to GitHub should run the Makefile test target (to ensure we only ever push working code) as part of a CI workflow.

Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

Some tests (via workflow) are done but as for running make test I don't think it's even possible in full. And the real one that needs to be verified is make prep but that would require a lot.

Agree.

On the other hand one can add a commit hook for their own that runs prior to committing.

That might be useful, @xexyl. Would that hook run make prep or make release or some other test?

Just as a reminder I did this once but you thought it might be the wrong time. Is it a good time now?

I can look and see if I still have what I did or else rewrite it.

BUT it might be possible to only have an example for people to use if they want to.

Let me know what you think.

It seems like now is the time. The code is reasonable stable.

If you propose a pull request, we can try it out before we declare (i.e., tag) the current contents "version 1.2" and make it a formal release.

[xexyl]

Each push to GitHub should run the Makefile test target (to ensure we only ever push working code) as part of a CI workflow.

Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

Some tests (via workflow) are done but as for running make test I don't think it's even possible in full. And the real one that needs to be verified is make prep but that would require a lot.

Agree.

On the other hand one can add a commit hook for their own that runs prior to committing.

That might be useful, @xexyl. Would that hook run make prep or make release or some other test?

Just as a reminder I did this once but you thought it might be the wrong time. Is it a good time now?

I can look and see if I still have what I did or else rewrite it.

BUT it might be possible to only have an example for people to use if they want to.

Let me know what you think.

It seems like now is the time. The code is reasonable stable.

If you propose a pull request, we can try it out before we declare (i.e., tag) the current contents "version 1.2" and make it a formal release.

Okay. Though like I said I don't know if it can be enforced.

Which rule should it run? Obviously not release but maybe prep?

[SirWumpus]

I suggest the CI workflow test as much as possible. A quick glance at the Makefile, I'd say targets: test, test-chkentry, check_man at the least.

Does make prep do all that?

[SirWumpus]

BTW my apologies if I put anyone's nose out of joint.

[xexyl]

BTW my apologies if I put anyone's nose out of joint.

Of course you didn't! Don't think about it. It's a good suggestion too.

It might be that one has to install it in their fork but I don't know. I am pretty sure that's how hooks work but maybe there's a way to do it so it's enforced.

[xexyl]

I suggest the CI workflow test as much as possible. A quick glance at the Makefile, I'd say targets: test, test-chkentry, check_man at the least.

Does make prep do all that?

Well make prep does quite a few things including running bug_report.sh (but with the option to delete the log file if no problems are detected - this is why I was asked to put it in iirc).

The release one cannot be put in because it forces one to have the right version of flex and bison but only people (Landon and me being the only ones .. at least so far?) working on the parser need that.

It's suggested that one runs make prep prior to committing though certainly we have neglected to do this only to cause a problem that had to be fixed.

I also am aware of a situation where the state of my fork let the prep process work but Landon got errors. I don't remember the details: that was quite some time back and lots has happened since then (not just here but other things in life of course).

But I don't really know much about GitHub workflows either.

I just got home from the doctor a short bit ago and I am going to turn the laptop on shortly. I can look into it I think though there are some other things I need to take care of first.

Please feel free to offer suggestions on how this might work!

[SirWumpus]

I'd prefer this as much of this as a GitHub workflow (vs a .git/hooks/pre-commit) so that pushes and pull-requests are always tested. Relying on each contributing developer to apply a pre-commit hook and not skip it (git commit -n) is taking a lot on trust.

[xexyl]

I'd prefer this as much of this as a GitHub workflow (vs a .git/hooks/pre-commit) so that pushes and pull-requests are always tested. Relying on each contributing developer to apply a pre-commit hook and not skip it (git commit -n) is taking a lot on trust.

It is indeed. The question is whether it will work. If it's run on GitHub it won't have access to all the tools needed.

There is a workflow but it doesn't run make prep or make test even and I am not sure how it could.

But if it's possible I don't know how. Do you have any idea?

Booting up laptop now and will take care of those other things then I will look at this.

[SirWumpus]

Well I just added to my https://github.com/SirWumpus/iocccsize repo a CI workflow, which is probably the simplest.

I'm now trying to add one to https://github.com/SirWumpus/post4, which wants OpenJDK 17+ if possible which is a little more complex. Add to the fact that the Ubuntu gcc used for the workflow complains about stuff I've not had issue with before (printf formatting- actually had issues and fixed, but now its even more pedantic and annoying and have to revisit it - I keep telling myself its all in the name of portablity).

[SirWumpus]

If it's run on GitHub it won't have access to all the tools needed.

It is possible to have tools added to the environment. I once had a to deal with Erlang testing.

Based on the setup-java example I've just been playing with, you should find one for parser tools, or tools in general. It will then require use: statement blocks and with: to control options. There should be some ready made related examples you can search for.

[xexyl]

If it's run on GitHub it won't have access to all the tools needed.

It is possible to have tools added to the environment. I once had a to deal with Erlang testing.

Based on the setup-java example I've just been playing with, you should find one for parser tools, or tools in general. It will then require use: statement blocks and with: to control options. There should be some ready made related examples you can search for.

Can you add tools that are part of the repo? They have to be compiled by the Makefile.

If that's possible that seems like a good idea; otherwise I don't know what to do. Is it? Did you find some documentation that can be looked at to do this? If there is documentation I can look into it and maybe figure it out.

Thanks!

[SirWumpus]

Would you like me to take this task on (since I've touch it a little in the past)? It should allow you to get on with other things and give me something else to do other than "proof documentation" and raise tickets.

[xexyl]

Would you like me to take this task on (since I've touch it a little in the past)? It should allow you to get on with other things and give me something else to do other than "proof documentation" and raise tickets.

If you'd like to you certainly could and it'd be helpful indeed.

The right one to run is make prep. You might try running it on your system first to see what it does. It's pretty extensive.

If you have any questions Landon or I can happily answer and if it proves to be a problem and you can link to some documentation I can tackle it later on.

Thanks!

[SirWumpus]

Right then, assign it to me and I'll get stuck in.

[xexyl]

Right then, assign it to me and I'll get stuck in.

I of course cannot do that. Landon?

[lcn2]

Just as a reminder I did this once but you thought it might be the wrong time. Is it a good time now?

As that time, the code was under significant development and did not pass tests. Moreover the test was under development as well. One could argue that the tests should have been ready from the start: that wasn't the case back then. :-)

Comments for @SirWumpus

make prep and make release

As far as doing "GitHub CI / CD workflow", @SirWumpus, we have the make prep and make release rule such that if it exit 0 then all is well and it it exit != 0 then something went wrong. The tree is left in a compiled state, so perhaps if all is well then a make clobber needs to run in order to restore the tree to a "just the source" state.

One difference between make prep differs from make release is that prep will note errors and fail at the end whereas release will exit at the first error. Both make rules are driven by test_ioccc/prep.sh, and so:

$ test_ioccc/prep.sh -h
usage: test_ioccc/prep.sh [-h] [-v level] [-V] [-e] [-o] [-m make] [-M Makefile] [-l logfile]

    -h              print help and exit
    -v level        flag ignored
    -V              print version and exit

    -e          exit on first make action error (def: exit only at end)
    -o          do NOT use backup files, fail if bison or flex cannot be used (def: use)
    -m make     make command (def: make)
    -M Makefile     path to Makefile (def: ./Makefile)
    -l logfile      write details of actions to logfile (def: send to stdout)

Exit codes:
     0   all OK
     1   -h and help string printed or -V and version string printed
     2   command line error
     3   Makefile not a readable file
     4   could not make writable log file
 >= 10   some make action exited non-zero

prep.sh version: 1.0.1 2024-03-02

Another key difference is between make prep differs from make release is if -o is supplied to test_ioccc/prep.sh. Under release -o is supplied while under prep it is not.

And what is the deal about -o? We do not require someone to have the proper level of flex(1) or lex(1) NOR do we require someone to have proper level of bison(1) or yacc(1). In both cases a minimum version is required to process jparse/jparse.l and jparse/jparse.y. We pre-build the result so that if someone lacks proper level of flex(1) or lex(1) and/or lacks the proper level of bison(1) or yacc(1) the code will simply C compile the pre-built parser code.

Given that, make release only is useful when run from a "reference development platform" where, among other things, we have a reasonably up to date flex(1) or lex(1) and a reasonably up to date bison(1) or yacc(1).

Development vs. Build

There are other dependencies that "reference development platform" needs to use make release, such as a reasonably up to date version of bash(1), etc. We also require a GNU Make-style make command as well as bash.

Passing make release is NOT supported on all platforms: only on "development platforms" that meet the requirements as noted above, among other things.

Passing make release is only needed by mkiocccentry repo developers and not someone who wants to submit the IOCCC.

See also comment-2205419620.

On client side or server side test hooks

This brings up the question of what mkiocccentry repo developer needs to do to react to make prep or make release failure: a mkiocccentry repo developer would need to read the logs, to make the corrections and to try again.

What sort of context (OS release, platform, etc.) would the "GitHub CI / CD workflow" be run under? We could see an advantage in having a commit tested under some platforms as part of the workflow. And how would a mkiocccentry repo developer make use / access the logs that were generated? And what if the problem occurs on the GitHub for a platform that is not interested in being supported?

In general, what does a GitHub CI / CD workflow buy us?

[xexyl]

Just as a reminder I did this once but you thought it might be the wrong time. Is it a good time now?

As that time, the code was under significant development and did not pass tests. Moreover the test was under development as well. One could argue that the tests should have been ready from the start: that wasn't the case back then. :-)

Comments for @SirWumpus

In general, what does a GitHub CI / CD workflow buy us?

I of course said some of that. Indeed very few people need make release. And some of the other tools that make prep run might not be strictly necessary for everyone.

I wonder very much also what the workflow might buy us. If it is not thought worth having them after all I can get the hooks I wrote and maybe SUGGEST people use them but not require them. That's also a possibility I guess.

But I must go to sleep now. Can barely see. Good night!

[lcn2]

Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

This repo does not have manifests, nor tarballs.

Perhaps you are thinking of some other repo?

[xexyl]

Unclear if a CD workflow should be added (never done one) to regenerate manifests, tarballs, snowballs, etc.

This repo does not have manifests, nor tarballs.

Perhaps you are thinking of some other repo?

I wish it had some snowballs though!

'I don't like this at all,' panted Cody just behind. 'Snow's all right on a fine morning, but I like to be in bed while it's falling. I wish this lot would go off to Hobbiton! Folk might welcome it there.'

Okay I'm not in Hobbiton and I replaced Sam with Cody but I'm in a part of the world that has awful temperatures but I'm not willing to say exactly where that is so it'll have to be Hobbiton :-) ... actually I haven't seen snow touch the ground here since the 1989-1990 winter or maybe it was the 1990-1991 winter ... and it was there for not even an hour and it was hardly any at all. I don't want it to snow but I'd certainly prefer colder weather than what we have now!

[xexyl]

And since I'm into 'The Ring Goes South' I might as well add another amusing thing Sam says:

'And it is no good going back while the storm holds,' said Aragorn. 'We have passed no place on the way up that offered more shelter than this cliff-wall we are under now.'

'Shelter!' muttered Sam. 'If this is shelter, then one wall and no roof make a house.'

.. okay I'll stop spouting LR references now :-)

[SirWumpus]

When I lived in Cannes, France snow at sea level is VERY rare around the Med.

[xexyl]

When I lived in Cannes, France snow at sea level is VERY rare around the Med.

Will look at your page shortly.

My elevation is somewhere between 210-300m but of course that is obviously changing :(

[xexyl]

When I lived in Cannes, France snow at sea level is VERY rare around the Med.

Will look at your page shortly.

Nice! Thanks for sharing. Hard for me to see with the phone but what I see looks nice and I can look later (or zoom way in of course).

[xexyl]

When I lived in Cannes, France snow at sea level is VERY rare around the Med.

Will look at your page shortly.

Nice! Thanks for sharing. Hard for me to see with the phone but what I see looks nice and I can look later (or zoom way in of course).

I LOVE black and white photos btw. Looked at them with laptop: very nice! Thanks for sharing.

[SirWumpus]

I LOVE black and white photos btw

Actually they're all colour; its just the day is overcast and the snow hides the green on the (palm) trees etc.

[xexyl]

I LOVE black and white photos btw

Actually they're all colour; its just the day is overcast and the snow hides the green on the (palm) trees etc.

Interesting .. though I often have a problem distinguishing colours. Thanks for clarifying: nice pictures either way!

[SirWumpus]

The CI workflow is complete. It will prepare 3rd party tools and run make slow_prep (easier to see all log output in case of an error via GitHub) and also make test. See pull request. Close if accepted.

[xexyl]

The CI workflow is complete. It will prepare 3rd party tools and run make slow_prep (easier to see all log output in case of an error via GitHub) and also make test. See pull request. Close if accepted.

No need to run make test if make prep or variation is run. It runs bug_report.sh which runs make test. So you might want to take that part out to save time.

It also runs make test separately btw.

[SirWumpus]

Well I found that you cannot run make test after make slow_prep without a clobber and rebuild oddly, so running it separately doesn't hurt and gives me warm fuzzies.

[xexyl]

Well I found that you cannot run make test after make slow_prep without a clobber and rebuild oddly, so running it separately doesn't hurt and gives me warm fuzzies.

Maybe but the prep scripts do that all several times so there's no real point and just slows down the process. Well we'll see anyway. Thanks for your work! I'll be happy to test a small commit when it's merged. But for now I must leave. Good day!

[lcn2]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

[lcn2]

We plan to submit a commit that should break the server side test. We will mark it with an appropriate XXX and see what happens and then revert as needed.

[xexyl]

We plan to submit a commit that should break the server side test. We will mark it with an appropriate XXX and see what happens and then revert as needed.

So .. how did it go?

I am off to bed but tomorrow I hope to finish the spoiler/obfuscation thing and then go back to fixing formatting of code blocks up before I then go back to looking at the rest.

I will try making a comment about the FAQ reorganising too.

Have a great night, both of you, and thank you Anthony also for doing this work. I really appreciate it. Saved me having to research it and I got other things that have to be done done (or mostly done).

Cheers.

[xexyl]

Well I found that you cannot run make test after make slow_prep without a clobber and rebuild oddly, so running it separately doesn't hurt and gives me warm fuzzies.

Quickly: as far as being unable to do make test without compiling. I thought that was strange too but it was decided to have it that way so I undid it.

It's true that make prep does make test and runs bug_report.sh which also runs make test but perhaps it's better this way too as it's another layer of defence.

I am curious how this looks as far as if a mistake is made: how will GitHub deal with it and how does the person fix it.

Well we will see I guess. Thanks again! Nn.

[SirWumpus]

I am curious how this looks as far as if a mistake is made: how will GitHub deal with it and how does the person fix it.

Pretty straight forward. You push, GitHub accepts, CI runs and either complains (sends email if you don't hang around for the results) that CI failed, or remains silent on success. The push to GitHub still happens. As I recall there will be some flag or indicator (or was that GitLab).

Same for a pull-request.

A person fixes it like any bug, through the typical ebb and flow of key strokes, clicky mice, until 9 minutes later (was going to say 9 months, but things move fast with computers) and update gets pushed again, GitHub accepts, runs CI ...

(Sorry little tired myself).

[lcn2]

So .. how did it go?

After issue #895:

We pre-checked the action by hand on the command line to verify that it could finish without error.

With commit 8ab316999240c97787d752a26bd82dfd5d86eada we added some text to CHANGES.md and updated version for test_ioccc/prep.sh.

The test.yml actions ran successfully.

We noted that for codeql.yml that we were using deprecated versions, so we issued commit 0ecac765aa7a105f6fb06dcd285d5c0972bebc27 to update ... this was the diff:

diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
index 3d3ab771..41372fa7 100644
--- a/.github/workflows/codeql.yml
+++ b/.github/workflows/codeql.yml
@@ -40,11 +40,15 @@ jobs:

     steps:
     - name: Checkout repository
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
+    - name: Setup node
+      uses: actions/setup-node@v4
+      with:
+        node-version: '20'

     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v2
+      uses: github/codeql-action/init@v3
       with:
         languages: ${{ matrix.language }}
         # If you wish to specify custom queries, you can do so here or in a config file.
@@ -58,7 +62,7 @@ jobs:
     # Autobuild attempts to build any compiled languages  (C/C++, C#, Go, or Java).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@v2
+      uses: github/codeql-action/autobuild@v3

     # ℹ️ Command-line programs to run using the OS shell.
     # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -71,6 +75,6 @@ jobs:
     #   ./location_of_script_within_repo/buildscript.sh

     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v2
+      uses: github/codeql-action/analyze@v3
       with:
         category: "/language:${{matrix.language}}"

Both the codeql.yml AND test.yml actions ran successfully.

Next TODO: Deliberately introduce an error that will cause the test.yml actions to fail and see what happens.

[lcn2]

QUESTION

Is the codeql.yml file current and best practices, or should be be improved?

See also comment-2208178415.

UPDATE 0

With commit ac85cb2587ef7fc8803d347ce377183248206184 we added something that calc had: .github/workflows/dependabot.yml

[lcn2]

We are going to introduce a deliberate error to make test:

diff --git a/Makefile b/Makefile
index 85d01c05..d3ea5197 100644
--- a/Makefile
+++ b/Makefile
@@ -915,6 +915,7 @@ test:
        ${E} ${MAKE} ${MAKE_CD_Q} -C test_ioccc $@
        ${E} ${RM} -f jparse/test_jparse/pr_jparse_test
        ${S} echo
+       exit 1 # XXX - deliberate failure to test code test.yml - XXX
        ${S} echo "${OUR_NAME}: make $@ ending"

 # run test-chkentry on test_JSON files

BTW: This will cause make slow_prep to fail with:

test_ioccc: make test ending
rm -f jparse/test_jparse/pr_jparse_test

exit 1 # XXX - deliberate failure to test code test.yml - XXX
make[1]: *** [test] Error 1
./test_ioccc/prep.sh: Warning: EXIT_CODE is now: 31

./test_ioccc/prep.sh: Warning: /Library/Developer/CommandLineTools/usr/bin/make -f ./Makefile test exit status: 2

=-=-= FAIL: /Library/Developer/CommandLineTools/usr/bin/make test =-=-=

=-=-=-=-= FAIL: ./test_ioccc/prep.sh =-=-=-=-=

=-=-=-=-= Will exit: 31 =-=-=-=-=

One or more tests failed:

    make_action 31: /Library/Developer/CommandLineTools/usr/bin/make -f ./Makefile test: non-zero exit code: 2

See test_ioccc/test_ioccc.log for more details.

make slow_prep: ERROR: ./test_ioccc/prep.sh exit code: 31

make slow_prep: see build.log for details
make: *** [slow_prep] Error 31

Now commit 73a965d0e8d42b6f20a727524cdd65d417f1ab07 introduces this error.

We see that the test.yml fails with:

One or more tests failed:

    make_action 31: make -f ./Makefile test: non-zero exit code: 2

See test_ioccc/test_ioccc.log for more details.

make slow_prep: ERROR: ./test_ioccc/prep.sh exit code: 31

make slow_prep: see build.log for details
make: *** [Makefile:579: slow_prep] Error 31
Error: Process completed with exit code 2.

QUESTION

From what we can see. the commit 73a965d0e8d42b6f20a727524cdd65d417f1ab07 is in the code. The test rule in Makefile line 918 shows the commit 73a965d0e8d42b6f20a727524cdd65d417f1ab07 line under the test rule:

        exit 1 # XXX - deliberate failure to test code test.yml - XXX

Now is it true that the action reports:

Some checks were not successful

However, as developers we run actions such as make release or make prep BEFORE doing a commit, so we would have caught this "error" beforehand anyway.

What is test.yml doing for us that we didn't / don't / won't already know? So, @SirWumpus, what did we gain from the test.yml file?

Yes, we can see the "red x" in the actions section. But we can also see that we have an "error" by doing any number of the make test variants before and after the commit.

PLEASE NOTE: There is NO sarcasm in this question. We do not intend for that to be a slight in any form. We are asking because we do not understand what has happened or what was gained from the test.yml file.

Or is there a bug in test.yml?

We will leave the make test error in the code for now, until we better understand what this new test.yml is doing for us.

Please advise, @SirWumpus - Thanks 🙏

[SirWumpus]

However, as developers we run actions such as make release or make prep BEFORE doing a commit, so we would have caught this "error" beforehand anyway.

Both of you probably do this manually before a commit and as mentioned a couple of times its possible to automate this per developer in their local .git/hooks/pre-commit, eg.

elf$ pwd
/home/achowe/git/com/snert/src/post4
elf$ cat .git/hooks/pre-commit
cd src && make test
elf$

It also possible that one day you are in a rush and skip the manual step. Or you explicitly skip it because you are committing WIP (work-in-progress) and it will fail, so you skip with git commit --no-verify.

These are all local to each developer's repository. What happens if a new contributor comes along? Will they follow best practices?

So CI on the GitHub side and a last line of defence. With small teams [1..3] it might seem overkill.

Yes, we can see the "red x" in the actions section. But we can also see that we have an "error" by doing any number of the make test variants before and after the commit.

Again, you have the discipline to do this before a commit. And likewise @xexyl.

Myself, my focus was iocccsize so I only ever fussed with make test before a commit (never looked at or understood the purpose of prep, slow_prep, release). So what happens if my piece impacts other tools? I would not have thought to test other areas with different targets. With CI in place I'd at least get a warning that something broke (before you had a chance to add a comment to an issue ;-) ).

Now in the Actions section you drill down to see where the error occurs: select failed workflow, click X build to expand the output, expand the failed task, scroll through the output (the slow_prep is very very long).

Or is there a bug in test.yml?

Nope. It looks correct and did its job and caught the failure. Cudos for the detailed log output. Note though my first iteration I used make prep, which reports an error, but no build.log output so I couldn't find errors and the build.log would be lost each run. But at least would tell me to repeat the test locally.

Not sure what else I can say about this. GitHub doesn't make it very prominent (I think GitLab might have slightly larger cross / tick icons; their setup is similar but not the same UI). A test suite is great to catch regressions and good developers run tests often. CI just ensures tests happen at least once on a push or pull-request.

Pull-requests from unknown 3rd parties are probably where you will benefit most from CI.

[SirWumpus]

Premature.

[xexyl]

We noted that for codeql.yml that we were using deprecated versions, so we issued commit 0ecac76 to update ... this was the diff:

diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
index 3d3ab771..41372fa7 100644
--- a/.github/workflows/codeql.yml
+++ b/.github/workflows/codeql.yml
@@ -40,11 +40,15 @@ jobs:

     steps:
     - name: Checkout repository
-      uses: actions/checkout@v3
+      uses: actions/checkout@v4
+    - name: Setup node
+      uses: actions/setup-node@v4
+      with:
+        node-version: '20'

     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v2
+      uses: github/codeql-action/init@v3
       with:
         languages: ${{ matrix.language }}
         # If you wish to specify custom queries, you can do so here or in a config file.
@@ -58,7 +62,7 @@ jobs:
     # Autobuild attempts to build any compiled languages  (C/C++, C#, Go, or Java).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@v2
+      uses: github/codeql-action/autobuild@v3

     # ℹ️ Command-line programs to run using the OS shell.
     # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -71,6 +75,6 @@ jobs:
     #   ./location_of_script_within_repo/buildscript.sh

     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v2
+      uses: github/codeql-action/analyze@v3
       with:
         category: "/language:${{matrix.language}}"

I just synced this to the jparse repo .. for later. But of course if we get this issue sorted it'd be better to use something like this. But that being said I didn't see where the files are. That is I did not yesterday see files being added. Which files are added, @SirWumpus ? Thanks.

[xexyl]

Interesting thing with me syncing my fork with the mkiocccentry repo: https://github.com/xexyl/mkiocccentry/actions/runs/9793882998.

So it definitely caught something too though I'm too tired to know what now ... but it apparently works or does something anyway.

[xexyl]

However, as developers we run actions such as make release or make prep BEFORE doing a commit, so we would have caught this "error" beforehand anyway.

Both of you probably do this manually before a commit and as mentioned a couple of times its possible to automate this per developer in their local .git/hooks/pre-commit, eg.

elf$ pwd
/home/achowe/git/com/snert/src/post4
elf$ cat .git/hooks/pre-commit
cd src && make test
elf$

It also possible that one day you are in a rush and skip the manual step. Or you explicitly skip it because you are committing WIP (work-in-progress) and it will fail, so you skip with git commit --no-verify.

These are all local to each developer's repository. What happens if a new contributor comes along? Will they follow best practices?

Of course ... and we sometimes forget too. The hook can be useful of course but sometimes we might also need to get a fix in quickly and we might not have time to do that (rare but it has happened at least for me).

So CI on the GitHub side and a last line of defence. With small teams [1..3] it might seem overkill.

Yes, we can see the "red x" in the actions section. But we can also see that we have an "error" by doing any number of the make test variants before and after the commit.

Again, you have the discipline to do this before a commit. And likewise @xexyl.

Usually we do .. both of us have made mistakes :-)

Myself, my focus was iocccsize so I only ever fussed with make test before a commit (never looked at or understood the purpose of prep, slow_prep, release). So what happens if my piece impacts other tools? I would not have thought to test other areas with different targets. With CI in place I'd at least get a warning that something broke (before you had a chance to add a comment to an issue ;-) ).

The make prep is much more extensive indeed and covers a wide range of things: it makes sure everything works together.

Now in the Actions section you drill down to see where the error occurs: select failed workflow, click X build to expand the output, expand the failed task, scroll through the output (the slow_prep is very very long).

Or is there a bug in test.yml?

Nope. It looks correct and did its job and caught the failure. Cudos for the detailed log output. Note though my first iteration I used make prep, which reports an error, but no build.log output so I couldn't find errors and the build.log would be lost each run. But at least would tell me to repeat the test locally.

As for build.log .... that (if I'm thinking right) should be in the top level directory. Or are you saying that it was empty? Maybe it's in the test_ioccc subdirectory?

Not sure what else I can say about this. GitHub doesn't make it very prominent (I think GitLab might have slightly larger cross / tick icons; their setup is similar but not the same UI). A test suite is great to catch regressions and good developers run tests often. CI just ensures tests happen at least once on a push or pull-request.

And that seems like a good idea.

[xexyl]

As far as when I pushed to my fork ... this is what I got:

It seems that this repo got more details than that though. I don't know if this part is a problem or not of course.

[SirWumpus]

That view looks different. Oh you're looking at the dependabot.yml output. Instead on the LHS click C/C++ CI or All Workflows then on the RHS drill down to the error output: click in the failed workflow item, then the X Build (button?), which in turn expands to show the steps completed (with timings), then that should jump scroll to the error output. Of course you need to scroll back to see the actual location.

[xexyl]

That view looks different. Oh you're looking at the dependabot.yml output. Instead on the LHS click C/C++ CI or All Workflows then on the RHS drill down to the error output: click in the failed workflow item, then the X Build (button?), which in turn expands to show the steps completed (with timings), then that should jump scroll to the error output. Of course you need to scroll back to see the actual location.

I just clicked on the link in the email .. let's see. Ah. I was sent two emails. I see the more detailed view now. Thanks! Will look at it in more detail (or try to .. still not all that awake).

[xexyl]

Okay looked at it again: very nice! I think this will be helpful when jparse will be in its own repo.

Speaking of such, Landon, did you say this should be done after the fork merge? When I do it, anyway, we will have to do it together because it will require some (possibly significant) modification to the mkiocccentry repo too and it might be hard (at first) to import it all into its own repo.

But what do you think @lcn2? I think it will be nice to have it in its own repo and others could make use of it more too but it's not yet time I think. We were going to wait until the three tools (that have to be mostly rewritten I guess) were done but that has changed I think. Still it's not until after the fork merge, that you want this, right (if nothing else as we want to focus on that)?