ioccc-src / mkiocccentry

Form an IOCCC submission as a compressed tarball file
Other
28 stars 6 forks source link

Enhancement: Write the bug-report.sh tool #248

Closed lcn2 closed 1 year ago

lcn2 commented 2 years ago

Add a make report that would run the test suite and write various info to say bug-report.txt

The info would include version numbers, out of commands such as uname -a, compiler version info, contents of limit_ioccc.sh, encourage the user to report the contents of the bug-report.txt file in a GitHub bug report, etc.

xexyl commented 2 years ago

We lost about 45 minutes trying to type in detailed answers to all the above :( only to have GitHub toss our comment. :-(

Ugh! I am really sorry about that. This has happened to me so many times with different websites over the years! I have many times told myself that I will always reply in vim and copy paste it but I rarely actually do that.

I have also had git commit mess up where the log ends up being empty so I lost time with that too. Still not sure what causes that.

GitHub has caused me other problems as well. Anyway I am really sorry that it rejected your comment! That’s so typical and so bloody annoying! 🤮🧨

And with that I am going to put my phone down. Have a great night and again I am sorry that github wasted your time 😩

xexyl commented 2 years ago

If it helps btw: feel free to try and make quicker replies and if necessary I will ask for more information.

Of course there’s the hilarious irony where often trying to make messages shorter actually takes longer. Some humorists have even said they would try making their messages shorter if they had the time. It’s a very funny thing and as someone who writes a lot I understand it all too well. It really does take more time to write shorter messages and when it comes to technical things it might even be a mistake.

Nevertheless I would say try not worrying about it for now. Take a break maybe if you need one. There’s plenty I can reply to tomorrow.

Anyway I really will be going now. Good night my friend and I am sorry that that happened! Very frustrating and it at least for me always makes me wonder ‘why?’ - with of course not only no satisfactory answer but actually no answer at all!

Anyway if you need a break or whatever that’s fine. We will work this out in time. Good night now!

xexyl commented 2 years ago

A94EE712-77F3-48B3-875E-10DB0F7D721F

00E0222E-77AD-48F2-923D-DCEC016A4100

Since you might want some laughs after that. I was trying to attach other ones but on the phone with so many images it and being so tired I gave up. These will hopefully give you a few laughs though. Putting phone down now. Good night and I will reply to the comments tomorrow!

lcn2 commented 2 years ago

We are going to try an restate the purposes for bug-report and hostchk.sh in the hopes the these clarify things rather than to try and repeat detailed answers:

bug-report.sh

When a user complains about the mkiocccentry repo not working, if we are going to help the user we need information about BOTH their environment AND what went wrong.

Somehow we need to direct users who have problems to file a useful bug report. The bug-report.sh script should try to create a bug report with lots of useful information about the user's compile environment, compile experience, and the result of various tests that could be performed.

The user would be encouraged to mail the bug report, perhaps compressed, to a bug report Email address, with a suitable subject line indicating that this is a bug report.

Consider the case where some user tells you that your code does not work. What would you do?

You would probably ask th why what went wrong. Well the bug report should include what went wrong. So the bug-report.sh script might want to do:

make clobber
make all
make test

collecting the output into the bug report file.

You might want to also know if there is someone odd, missing or bogus with their system environment. So the bug-report.sh script might want to run the hostchk.sh script, with some level of verbosity, to help identify what might be wrong, missing, or odd about their environment. The output of that verbose hostchk.sh script should be included in the bug report file.

AGAIN, think about what YOU might want to know about a user who is having some sort of problem with the mkiocccentry repo. That problem could be anything from they do not have make(1), to a missing include file, to a compiler that is super old, to an actual bug in a tool such as txzchk, etc.

For cases where code is able to be compiled, your idea of saving the output of ./foo -h is a good one too. Of course, if a tool such as txzchk is missing (because of a compile problem) that should be noted. If txzchk exists, and ./txzchk -h does something crazy, that should be noted too.

hostchk.sh

The hostchk.sh script, by default, should be run early as part of a make all rule. The purpose is to raise a warning flag(s) about the user's system environment with respect to their ability to compile the code.

The hostchk.sh script should do things like check for C compiler executable being found, and printing the path to it.

The hostchk.sh script should run the compiler with the -v to get the compiler version.

The hostchk.sh script should try to compile a tiny C program that just exits:

int
main(void)
{
    return 0;
}

it should check that the compile was successful and then that the executable exits 0.

Then try:

int
main(void)
{
    return 1;
}

and check for exit 1.

Then try a basic Hello world:

#include <stdio.h>
int
main(void)
{
    (void) printf("Hello, world\n");
    return 0;
}

From there it can add various system include files, one at a time, such as:

#include <errno.h>
int
main(void)
{
    return 0;
}

checking that the file can compile and execute .. thus testing if the system include file exists.

For C non-compiler stuff, checking shell version is a good idea.

Printing the users environment, say via printenv is a good idea.

Looking for the make(1) command, and printing the make version is a good idea.

Including some rule such as:

printvars:
          echo "CC=${CC}"
          echo "C_STD=${C_STD}"
          echo "COPT=${COPT}"
          echo "WARN_FLAGS=${WARN_FLAGS}"
          ...

that prints the value of each Makefile variable is useful.

Printing the contents of makefile.local, or the absence of that file is useful.

Testing make(1) on a makefile simple makefile such as:

all:
          echo all is well

is useful.

Then try a tiny makefile that has a gnu-like directive such as:

ifeq ($(FOO),)
FOO="bar"
endif

all:
          echo FOO should be bar
          echo FOO is "${FOO}"

to see if their make(1) as problems with gnu-like directives is a good idea.

Then try a tiny makefile that has a gnu-like directive that invokes the shell such as:

ifeq ($(FOO),)
FOO=$(shell echo bazl)
endif

all:
          echo FOO should be bay
          echo FOO is "${FOO}"

etc.

Stuff like that.

xexyl commented 2 years ago

Great ideas!

I look forward to looking at it in more detail tomorrow. Going now for real. I am glad you got some laughs out of those images. I have some others in mind as well but they will have to wait for another time.

Good night my friend! Enjoy the rest of your day.

lcn2 commented 2 years ago

Regarding bug-report.sh and make bug-report

The make bug-report rule should run bug-report.sh.

The bug-report.sh would create and write file off the form bug-report-YYYYMMDD-HHMMSS.txt when then print instructions on how to send that bug report to us.

Regarding hostchk.sh and bug-report.sh

Because the output of hostchk.sh might be useful in debugging a problem, the output of hostchk.sh with a stable verbosity level, would be useful for bug-report.sh to execute and include in the output that is generates.

Regarding hostchk.sh and Makefile

For a make all (default make), the hostchk.sh should be run early on with -v 0. If hostchk.sh finds no concerns with the environment, then it should just quietly exit 0.

Perhaps:

hostchk: hostchk.sh
    @./hostchk.sh -f -v 0; status="$$?"; if [ "$status" -ne 0 ]; then \
        make hostchk_warning; \
    fi

hostchk_warning:
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! hostchk exited non-zero !!' 1>&2
    echo '!! your build environment may not be able to compile mkiocccentry and friends !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! try running:' 1>&2
    echo './hostchk.sh -v 1' 1>&2
    echo 'for hints as to what might be wrong !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! If you think this is a bug, consider filing a bug report via:' 1>&2
    echo './bug-report.sh' 1>&2
    echo '!! about to sleep 10 seconds !! 1>&2
    sleep 10
    echo '!! letting the compile continue in hopes it might be OK, but we doubt in will !! 1>&2

The purpose of make hostchk_warning is to get the attention of the user, via a brief but slightly painful error message that includes hints as to what to do next.

UPDATE 0

Changed the proposed make hostchk rule to be silent by default if nothing is wrong, and use the -f flag to include fast check. See the next comment for -f.

lcn2 commented 2 years ago

If we are going to force the default compile (via make all) to run hostchk.sh then the execution of hostchk.sh should be very quick.

Perhaps the make all should use a special flag such as -f for fast check.

For example, instead of testing, one at a file, the ability to compile a tiny C program with each required system include file, the -f mode should try a all at once compiler such as:

#include <ctype.h>
#include <errno.h>
#include <float.h>
#include <inttypes.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/types.h>
#include <time.h>
...
int
main(void)
{
    return 0;
}

So that with 1 compile, all of the required system include files can be checked.

Without -f, each required system include file can be tried one at a time, allowing the hostchk.sh script to print a specific warning such as:

hostchk.sh: Warning: unable to compile with system include file: stdint.h

Anyway, consider the -f option to be the fast path, exiting 0 and printing nothing of all appears well.

Remember that we, and all others, will have to endure the time it takes for hostchk.sh -f to run when compiling the repo!

If the first make file is make all then perhaps:

######################################
# all - default rule - must be first #
######################################

all:
      make fast_hostchk
      make just_all

# The original make all that bypasses running hostchk.sh
# 
just_all: ${TARGETS} ${TEST_TARGETS}

# fast build environment sanity check 
#
fast_hostchk: hostchk.sh
    @./hostchk.sh -f -v 0; status="$$?"; if [ "$status" -ne 0 ]; then \
        make hostchk_warning; \
    fi

# slower more verbose build environment sanity check 
#
hostchk: hostchk.sh
    @./hostchk.sh -v 1; status="$$?"; if [ "$status" -ne 0 ]; then \
        make hostchk_warning; \
    fi

# get the users attention when hostchk.sh finds a problem
#
hostchk_warning:
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! hostchk exited non-zero !!' 1>&2
    echo '!! your build environment may not be able to compile mkiocccentry and friends !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! try running:' 1>&2
    echo './hostchk.sh -v 1' 1>&2
    echo 'for hints as to what might be wrong !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! If you think this is a bug, consider filing a bug report via:' 1>&2
    echo './bug-report.sh' 1>&2
    echo '!! about to sleep 10 seconds !! 1>&2
    sleep 10
    echo '!! letting the compile continue in hopes it might be OK, but we doubt in will !! 1>&2

Sans typos .. something like that.

lcn2 commented 2 years ago

If there is something that is still not clear, or if there is an important question you asked previously that has not been successfully addressed, please ask again, one question per comment below.

UDPATE 0

We are trying to clear the backlog of GitHub comments in order to return to finishing the JSON semantic code, plus creating a major revision of the web site. As such we make have missed a critical question. If so, please ask again, one question per comment below.

lcn2 commented 2 years ago

See commit 89b0c18cc966e42fa86b5548c333807d25a5c0c2 for an initial implementation for comment 1276865756

lcn2 commented 2 years ago

Changed use #248 to "Enhancement: Write the bug-report.sh tool".

We can add a make bug-report rule if needed, however the majority of the work will be in the bug-report.sh shell script.

For cases where the environment cannot use the Makefile, bug-report.sh is needed.

xexyl commented 2 years ago

Changed use #248 to "Enhancement: Write the bug-report.sh tool".

Just added a stub file. However I made it an underscore to match the rest of the filenames. Maybe you'll want to edit the title to this post too? Anyway see 727ef87586d020c00038b0f83224ce5529c5a314.

We can add a make bug-report rule if needed, however the majority of the work will be in the bug-report.sh shell script.

That might be a good idea to add rule. Perhaps you can do it since you apparently have some ideas of what it should do besides running the script itself.

For cases where the environment cannot use the Makefile, bug-report.sh is needed.

For example? I would hate to not be able to use the Makefile - so many tools are needed! Well anyway a stub file has been added. When we discuss it more I can start adding code.

xexyl commented 2 years ago

If there is something that is still not clear, or if there is an important question you asked previously that has not been successfully addressed, please ask again, one question per comment below.

UDPATE 0

We are trying to clear the backlog of GitHub comments in order to return to finishing the JSON semantic code, plus creating a major revision of the web site. As such we make have missed a critical question. If so, please ask again, one question per comment below.

No worries. And no rush either!

xexyl commented 2 years ago

We are going to try an restate the purposes for bug-report and hostchk.sh in the hopes the these clarify things rather than to try and repeat detailed answers:

Good idea and lots of good stuff here!

bug-report.sh

When a user complains about the mkiocccentry repo not working, if we are going to help the user we need information about BOTH their environment AND what went wrong.

Makes sense. I'll refer to bug_report.sh from now on for the reasons cited above.

Somehow we need to direct users who have problems to file a useful bug report. The bug-report.sh script should try to create a bug report with lots of useful information about the user's compile environment, compile experience, and the result of various tests that could be performed.

Of course.

The user would be encouraged to mail the bug report, perhaps compressed, to a bug report Email address, with a suitable subject line indicating that this is a bug report.

Although email might be one option wouldn't GitHub issues page be the preferable method? Consider if they have a system with a different format listing of tar. I would need to update txzchk but if it's over email I'm not going to see it and it would be better (I think?) if they could have the log on the website?

Consider the case where some user tells you that your code does not work. What would you do?

Ah yes .. the 'it doesn't work!' bug reports. Just wonderful! :-)

You would probably ask th why what went wrong. Well the bug report should include what went wrong. So the bug-report.sh script might want to do:

make clobber
make all
make test

collecting the output into the bug report file.

Good starting points. See https://github.com/ioccc-src/mkiocccentry/pull/362/commits/8d949c7b7d2c1ec791c4145f3d845db115d86dee please. I have some thoughts about this which I'll say below. Then I'll reply to the rest of this comment in another comment.

NOTE

Although I feel these are all related and a single point if you feel that this would be better split up into multiple comments please let me know and I'd be happy to do that - though it might take a bit of work to figure out how to do so.

--

There is something to consider here that I thought of. In the make test invocation some of the scripts run write to their own log file and these are not going to be in the log file. Some might or might not be deleted by their respective script even if they're needed in our new bug report script.

This might be important. Now this can be done differently but if one is to rely on make test then it would require some changes to the script ioccc_test.sh I think and possibly quite a few changes. For instance let's say that txzchk_test.sh fails on their end. Where do we record that? We could change the ioccc_test.sh script to have a mode for each test that has a log file to cat the file but is this the right approach? Another way that might be better is for the bug_report.sh script to (after running all the tests e.g. via make test) to concatenate the logs of the scripts that were created. But not all of this is clear either.

Okay but this is more complicated: if txzchk_test.sh fails it suggests what commands to run (which have to be updated in the script but I'll do that another time). The log file itself actually will not show what was reported - I think - but rather that it failed and for what the user to try. This might not be enough information for a bug report though.

What do you think about these?

xexyl commented 2 years ago

You might want to also know if there is someone odd, missing or bogus with their system environment. So the bug-report.sh script might want to run the hostchk.sh script, with some level of verbosity, to help identify what might be wrong, missing, or odd about their environment. The output of that verbose hostchk.sh script should be included in the bug report file.

Well they certainly won't be as odd as me! :-) .. No, I know what you mean. You're right. I actually forgot to have it run this script in the first commit. I have to do that but later on. I think I'm only going to reply to the comments and then take a break and do something else. Might not return to it today. Not sure. I'm feeling like I won't though.

This does bring up an interesting point though. If say make test runs all these tests but we need more details would it be better to just have the bug report script actually run the tests manually with a higher verbosity?

AGAIN, think about what YOU might want to know about a user who is having some sort of problem with the mkiocccentry repo. That problem could be anything from they do not have make(1), to a missing include file, to a compiler that is super old, to an actual bug in a tool such as txzchk, etc.

Speaking of such: should the script allow for passing in the paths to these tools?

For cases where code is able to be compiled, your idea of saving the output of ./foo -h is a good one too. Of course, if a tool such as txzchk is missing (because of a compile problem) that should be noted. If txzchk exists, and ./txzchk -h does something crazy, that should be noted too.

Ah .. so that might be why not to just grep the version? I was going to ask about that. What should we look for? I mean we could do an exact match but this seems like it might be overkill?

hostchk.sh

The hostchk.sh script, by default, should be run early as part of a make all rule. The purpose is to raise a warning flag(s) about the user's system environment with respect to their ability to compile the code.

The hostchk.sh script should do things like check for C compiler executable being found, and printing the path to it.

The hostchk.sh script should run the compiler with the -v to get the compiler version.

The hostchk.sh script should try to compile a tiny C program that just exits:

int
main(void)
{
    return 0;
}

it should check that the compile was successful and then that the executable exits 0.

Then try:

int
main(void)
{
    return 1;
}

and check for exit 1.

Then try a basic Hello world:

#include <stdio.h>
int
main(void)
{
    (void) printf("Hello, world\n");
    return 0;
}

From there it can add various system include files, one at a time, such as:

#include <errno.h>
int
main(void)
{
    return 0;
}

checking that the file can compile and execute .. thus testing if the system include file exists.

Good calls. I will have to do that another day though but it does answer the question of whether or not hostchk needs any updates.

For C non-compiler stuff, checking shell version is a good idea.

Printing the users environment, say via printenv is a good idea.

Looking for the make(1) command, and printing the make version is a good idea.

Do you think this should be in hostchk too? It seems like it'd be better there but since this is the thread for bug_report.sh I'm going to ask.

Including some rule such as:

printvars:
          echo "CC=${CC}"
          echo "C_STD=${C_STD}"
          echo "COPT=${COPT}"
          echo "WARN_FLAGS=${WARN_FLAGS}"
          ...

that prints the value of each Makefile variable is useful.

That's an idea.

Printing the contents of makefile.local, or the absence of that file is useful.

Testing make(1) on a makefile simple makefile such as:

all:
          echo all is well

is useful.

Then try a tiny makefile that has a gnu-like directive such as:

ifeq ($(FOO),)
FOO="bar"
endif

all:
          echo FOO should be bar
          echo FOO is "${FOO}"

to see if their make(1) as problems with gnu-like directives is a good idea.

Then try a tiny makefile that has a gnu-like directive that invokes the shell such as:

ifeq ($(FOO),)
FOO=$(shell echo bazl)
endif

all:
          echo FOO should be bay
          echo FOO is "${FOO}"

etc.

Stuff like that.

This makes me think that maybe there ought to be several more files. What do you think about that? I'm kind of thinking along the lines of test_iocccsize/. Should we have a directory for hostchk too? I'm starting to think that would be a good idea.

xexyl commented 2 years ago

Regarding bug-report.sh and make bug-report

The make bug-report rule should run bug-report.sh.

The bug-report.sh would create and write file off the form bug-report-YYYYMMDD-HHMMSS.txt when then print instructions on how to send that bug report to us.

This is done already though there's A LOT that has to be done and the first thing that has to be done is fix an issue that kind of makes the check for exit codes pointless.

Regarding hostchk.sh and bug-report.sh

Because the output of hostchk.sh might be useful in debugging a problem, the output of hostchk.sh with a stable verbosity level, would be useful for bug-report.sh to execute and include in the output that is generates.

Makes sense though I realised a problem with the initial code for bug_report.sh - I'll get to that in another comment soon.

Regarding hostchk.sh and Makefile

For a make all (default make), the hostchk.sh should be run early on with -v 0. If hostchk.sh finds no concerns with the environment, then it should just quietly exit 0.

Perhaps:

hostchk: hostchk.sh
    @./hostchk.sh -f -v 0; status="$$?"; if [ "$status" -ne 0 ]; then \
        make hostchk_warning; \
    fi

hostchk_warning:
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! hostchk exited non-zero !!' 1>&2
    echo '!! your build environment may not be able to compile mkiocccentry and friends !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! try running:' 1>&2
    echo './hostchk.sh -v 1' 1>&2
    echo 'for hints as to what might be wrong !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! If you think this is a bug, consider filing a bug report via:' 1>&2
    echo './bug-report.sh' 1>&2
    echo '!! about to sleep 10 seconds !! 1>&2
    sleep 10
    echo '!! letting the compile continue in hopes it might be OK, but we doubt in will !! 1>&2

The purpose of make hostchk_warning is to get the attention of the user, via a brief but slightly painful error message that includes hints as to what to do next.

UPDATE 0

Changed the proposed make hostchk rule to be silent by default if nothing is wrong, and use the -f flag to include fast check. See the next comment for -f.

I see also you put that in a Makefile rule. I meanwhile added the code for -f option, noting that it's no longer as fast. :-)

xexyl commented 2 years ago

If we are going to force the default compile (via make all) to run hostchk.sh then the execution of hostchk.sh should be very quick.

Perhaps the make all should use a special flag such as -f for fast check.

For example, instead of testing, one at a file, the ability to compile a tiny C program with each required system include file, the -f mode should try a all at once compiler such as:

#include <ctype.h>
#include <errno.h>
#include <float.h>
#include <inttypes.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/types.h>
#include <time.h>
...
int
main(void)
{
    return 0;
}

So that with 1 compile, all of the required system include files can be checked.

Done. The script still collects the necessary header files of course like I did for the one-at-a-time method.

Without -f, each required system include file can be tried one at a time, allowing the hostchk.sh script to print a specific warning such as:

hostchk.sh: Warning: unable to compile with system include file: stdint.h

Which of course it does.

Anyway, consider the -f option to be the fast path, exiting 0 and printing nothing of all appears well.

Exiting 0 at least if the other tests also do not fail.

Remember that we, and all others, will have to endure the time it takes for hostchk.sh -f to run when compiling the repo!

Of course. It's a good call and the implementation of it should suffice I think.

If the first make file is make all then perhaps:

######################################
# all - default rule - must be first #
######################################

all:
      make fast_hostchk
      make just_all

# The original make all that bypasses running hostchk.sh
# 
just_all: ${TARGETS} ${TEST_TARGETS}

# fast build environment sanity check 
#
fast_hostchk: hostchk.sh
    @./hostchk.sh -f -v 0; status="$$?"; if [ "$status" -ne 0 ]; then \
        make hostchk_warning; \
    fi

# slower more verbose build environment sanity check 
#
hostchk: hostchk.sh
    @./hostchk.sh -v 1; status="$$?"; if [ "$status" -ne 0 ]; then \
        make hostchk_warning; \
    fi

# get the users attention when hostchk.sh finds a problem
#
hostchk_warning:
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! hostchk exited non-zero !!' 1>&2
    echo '!! your build environment may not be able to compile mkiocccentry and friends !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! try running:' 1>&2
    echo './hostchk.sh -v 1' 1>&2
    echo 'for hints as to what might be wrong !!' 1>&2
    echo '!! WARNING WARNING WARNING !!' 1>&2
    echo '!! If you think this is a bug, consider filing a bug report via:' 1>&2
    echo './bug-report.sh' 1>&2
    echo '!! about to sleep 10 seconds !! 1>&2
    sleep 10
    echo '!! letting the compile continue in hopes it might be OK, but we doubt in will !! 1>&2

Sans typos .. something like that.

And I see you added this - or most of it if not all of it. Thank you.

xexyl commented 2 years ago

Okay now as for commit 8d949c7b7d2c1ec791c4145f3d845db115d86dee there is a bug that makes it not nearly as useful. I did think of it initially but it escaped my mind before I committed it. It's basically this. Take the script fragment:

echo "## RUNNING make test: " | tee -a -- "$LOG_FILE" 1>&2
make test | tee -a -- "$LOG_FILE"  1>&2
status=$?
if [[ "$status" -ne 0 ]]; then
    EXIT_CODE=12
    echo "make test failed with exit code $status: new exit code: $EXIT_CODE" | tee -a -- "$LOG_FILE"
fi

This is of course buggy. Yes it's very amusing to me that a script called bug_report.sh has a bug but as I noted in:

# XXX - FIXME
#
# Currently the exit codes will not be correctly tested. The idea is here but
# the exit codes will not be tested right so if for instance make test fails it
# will still say that all tests passed. This will have to be fixed to make the
# testing of the exit codes useful but this will come at a later date.
#
# Yes it's very amusing that a script called bug_report.sh has a bug but amusing
# as it would have been I cannot say this was actually intentional! :-) I had
# thought of it originally but I neglected to address it until after a commit.
# This bug will be fixed at a later date as I must go do other things now. Still
# better to have it after all as who can't use more laughs? :-)
#
# XXX - FIXME

I cannot say it was intentional as funny as that would have been! :-) I did think of it but as said I failed to address it before the commit. I'm glad I did though because it's funny and we can all use some laughs. Even so it obviously has to be fixed. So how do you suggest this is done?

I can think of some ideas but I need a break so I'll leave this comment at that. We can discuss the methods and then I can fix it at another date. I'll be doing something more mindless now but I am glad I got as much done as I did today. I'm admittedly surprised.

Good day!

lcn2 commented 2 years ago

Although email might be one option wouldn't GitHub issues page be the preferable method? Consider if they have a system with a different format listing of tar. I would need to update txzchk but if it's over email I'm not going to see it and it would be better (I think?) if they could have the log on the website?

Perhaps the term Email is incorrect. Filing a GitHub issue and attaching a bug report might be a better idea.

lcn2 commented 2 years ago

There is something to consider here that I thought of. In the make test invocation some of the scripts run write to their own log file and these are not going to be in the log file. Some might or might not be deleted by their respective script even if they're needed in our new bug report script.

This might be important. Now this can be done differently but if one is to rely on make test then it would require some changes to the script ioccc_test.sh I think and possibly quite a few changes. For instance let's say that txzchk_test.sh fails on their end. Where do we record that? We could change the ioccc_test.sh script to have a mode for each test that has a log file to cat the file but is this the right approach? Another way that might be better is for the bug_report.sh script to (after running all the tests e.g. via make test) to concatenate the logs of the scripts that were created. But not all of this is clear either.

Okay but this is more complicated: if txzchk_test.sh fails it suggests what commands to run (which have to be updated in the script but I'll do that another time). The log file itself actually will not show what was reported - I think - but rather that it failed and for what the user to try. This might not be enough information for a bug report though.

What do you think about these?

It sounds like a problem that needs to be addressed. What did you suggest?

lcn2 commented 2 years ago

This makes me think that maybe there ought to be several more files. What do you think about that? I'm kind of thinking along the lines of test_iocccsize/. Should we have a directory for hostchk too? I'm starting to think that would be a good idea.

That might be an idea to consider.

xexyl commented 2 years ago

Although email might be one option wouldn't GitHub issues page be the preferable method? Consider if they have a system with a different format listing of tar. I would need to update txzchk but if it's over email I'm not going to see it and it would be better (I think?) if they could have the log on the website?

Perhaps the term Email is incorrect. Filing a GitHub issue and attaching a bug report might be a better idea.

Just wanted to be sure.

xexyl commented 2 years ago

There is something to consider here that I thought of. In the make test invocation some of the scripts run write to their own log file and these are not going to be in the log file. Some might or might not be deleted by their respective script even if they're needed in our new bug report script. This might be important. Now this can be done differently but if one is to rely on make test then it would require some changes to the script ioccc_test.sh I think and possibly quite a few changes. For instance let's say that txzchk_test.sh fails on their end. Where do we record that? We could change the ioccc_test.sh script to have a mode for each test that has a log file to cat the file but is this the right approach? Another way that might be better is for the bug_report.sh script to (after running all the tests e.g. via make test) to concatenate the logs of the scripts that were created. But not all of this is clear either. Okay but this is more complicated: if txzchk_test.sh fails it suggests what commands to run (which have to be updated in the script but I'll do that another time). The log file itself actually will not show what was reported - I think - but rather that it failed and for what the user to try. This might not be enough information for a bug report though. What do you think about these?

It sounds like a problem that needs to be addressed. What did you suggest?

If I'm honest I really don't have any idea right now. Do you have any thoughts? I think in my case for today I'll say: 'I'll sleep on it'. I'm just going through comments real quick and then will be taking it easy for a bit - unwinding - and then get ready for sleep. But this one does seem to be a problem that needs to be addressed somehow. Problem is deciding which way or ways to go about it.

xexyl commented 2 years ago

There is something to consider here that I thought of. In the make test invocation some of the scripts run write to their own log file and these are not going to be in the log file. Some might or might not be deleted by their respective script even if they're needed in our new bug report script. This might be important. Now this can be done differently but if one is to rely on make test then it would require some changes to the script ioccc_test.sh I think and possibly quite a few changes. For instance let's say that txzchk_test.sh fails on their end. Where do we record that? We could change the ioccc_test.sh script to have a mode for each test that has a log file to cat the file but is this the right approach? Another way that might be better is for the bug_report.sh script to (after running all the tests e.g. via make test) to concatenate the logs of the scripts that were created. But not all of this is clear either. Okay but this is more complicated: if txzchk_test.sh fails it suggests what commands to run (which have to be updated in the script but I'll do that another time). The log file itself actually will not show what was reported - I think - but rather that it failed and for what the user to try. This might not be enough information for a bug report though. What do you think about these?

It sounds like a problem that needs to be addressed. What did you suggest?

If I'm honest I really don't have any idea right now. Do you have any thoughts? I think in my case for today I'll say: 'I'll sleep on it'. I'm just going through comments real quick and then will be taking it easy for a bit - unwinding - and then get ready for sleep. But this one does seem to be a problem that needs to be addressed somehow. Problem is deciding which way or ways to go about it.

I'm more uncertain about the bug I referred to in comment https://github.com/ioccc-src/mkiocccentry/issues/248#issuecomment-1277931351 so if you have any suggestions on that I'm open to them as well. The problem, I guess, is that the status is the value of the tee command. I haven't actually looked into that but if that's the case I'm not sure how to get the exit code of the command as well as writing the output to a file unless it's simply written to the file and not shown in output too - and then checked against.

Do you think the user needs to see the output of all these commands? If not and that's actually the problem it would greatly simplify the problem. But like I said I've not really looked into it. What are your thoughts?

xexyl commented 2 years ago

This makes me think that maybe there ought to be several more files. What do you think about that? I'm kind of thinking along the lines of test_iocccsize/. Should we have a directory for hostchk too? I'm starting to think that would be a good idea.

That might be an idea to consider.

Thank you for confirming. I'll look into it later on - tomorrow or in the coming days. The compiling with all the needed header files is good the way it's being done because it means if any header file is changed (new one needed or one no longer needed) it'll be picked up.

I know there are some other comments in this thread too that I guess hostchk should use. Is that right? I mean should the compiler version and flex/bison versions and make etc. all be in hostchk ? Or should it be in bug report ? I refer btw to this comment (for example): https://github.com/ioccc-src/mkiocccentry/issues/248#issuecomment-1153283608.

Though given it refers to hostchk.sh maybe it should not be in hostchk itself ? I don't know.

lcn2 commented 2 years ago

This makes me think that maybe there ought to be several more files. What do you think about that? I'm kind of thinking along the lines of test_iocccsize/. Should we have a directory for hostchk too? I'm starting to think that would be a good idea.

That might be an idea to consider.

Thank you for confirming. I'll look into it later on - tomorrow or in the coming days. The compiling with all the needed header files is good the way it's being done because it means if any header file is changed (new one needed or one no longer needed) it'll be picked up.

I know there are some other comments in this thread too that I guess hostchk should use. Is that right? I mean should the compiler version and flex/bison versions and make etc. all be in hostchk ? Or should it be in bug report ? I refer btw to this comment (for example): #248 (comment).

Well the bug-report.sh tool should produce info about flex/bison (path, version) if they exist.

The hostchk.sh doesn't need to check flex/bison because the Makefile and run_bison.sh and run_flex.sh mange that already.

lcn2 commented 2 years ago

Do you think the user needs to see the output of all these commands? If not and that's actually the problem it would greatly simplify the problem. But like I said I've not really looked into it. What are your thoughts?

Probably not. One could run iocccsize_test.sh directly, for example, with a -v level if they wanted more information.

lcn2 commented 2 years ago

I'm more uncertain about the bug I referred to in comment https://github.com/ioccc-src/mkiocccentry/issues/248#issuecomment-1277931351 so if you have any suggestions on that I'm open to them as well. The problem, I guess, is that the status is the value of the tee command. I haven't actually looked into that but if that's the case I'm not sure how to get the exit code of the command as well as writing the output to a file unless it's simply written to the file and not shown in output too - and then checked against.

In bash, PIPESTATUS is an array variable containing a list of exit status values from the processes in the most-recently-executed foreground pipeline.

For example:

command1 | command2 
echo "${PIPESTATUS[0]} ${PIPESTATUS[1]}"
echo "${PIPESTATUS[@]}"

More examples:

false | false | true 
echo "The exit status of 1st command ${PIPESTATUS[0]}, 2nd command ${PIPESTATUS[1]}, and 34d command ${PIPESTATUS[2]}
echo "${PIPESTATUS[@]}"

You can use this to determine exit status of something you also pipe into tee.

UPDATE 0

See the make release rule in the Makefile for another use of PIPESTATUS.

xexyl commented 2 years ago

I'm more uncertain about the bug I referred to in comment #248 (comment) so if you have any suggestions on that I'm open to them as well. The problem, I guess, is that the status is the value of the tee command. I haven't actually looked into that but if that's the case I'm not sure how to get the exit code of the command as well as writing the output to a file unless it's simply written to the file and not shown in output too - and then checked against.

In bash, PIPESTATUS is an array variable containing a list of exit status values from the processes in the most-recently-executed foreground pipeline.

For example:

command1 | command2 
echo "${PIPESTATUS[0]} ${PIPESTATUS[1]}"
echo "${PIPESTATUS[@]}"

More examples:

false | false | true 
echo "The exit status of 1st command ${PIPESTATUS[0]}, 2nd command ${PIPESTATUS[1]}, and 34d command ${PIPESTATUS[2]}
echo "${PIPESTATUS[@]}"

You can use this to determine exit status of something you also pipe into tee.

UPDATE 0

See the make release rule in the Makefile for another use of PIPESTATUS.

Thanks for the reminder of this! I've seen it before but I've never had to use it. This I'm sure will help. Hope you're having a nice sleep my friend. I was really sleepy quite early last night so I got a lot of sleep. Not the best quality sleep but I got a lot of sleep. I'm hoping to reply to the comments and then see how I feel. Hopefully I can work some on the scripts. I'd like to be able to sleep right now but I'm unable to but if I can't get back to sleep at least I got a lot.

xexyl commented 2 years ago

This makes me think that maybe there ought to be several more files. What do you think about that? I'm kind of thinking along the lines of test_iocccsize/. Should we have a directory for hostchk too? I'm starting to think that would be a good idea.

That might be an idea to consider.

Thank you for confirming. I'll look into it later on - tomorrow or in the coming days. The compiling with all the needed header files is good the way it's being done because it means if any header file is changed (new one needed or one no longer needed) it'll be picked up. I know there are some other comments in this thread too that I guess hostchk should use. Is that right? I mean should the compiler version and flex/bison versions and make etc. all be in hostchk ? Or should it be in bug report ? I refer btw to this comment (for example): #248 (comment).

Well the bug-report.sh tool should produce info about flex/bison (path, version) if they exist.

The hostchk.sh doesn't need to check flex/bison because the Makefile and run_bison.sh and run_flex.sh mange that already.

Sure but the question is should bug_report.sh check the version of the other tools .. or should it let hostchk.sh do that? Given that you say hostchk doesn't need to because the Makefile does I guess it means that if the Makefile had not (via the two scripts) it would have been the hostchk.sh script.

Of course I read it when I was still waking up and so I missed the first line. It seems like versions should be detected in bug_report.sh so maybe I'll do that and if necessary it can be moved over to hostchk.sh.

To clarify why I think first of hostchk.sh: it's because it does a check on the host and checking the version on the host seems like a task for such a script. If however it's supposed to run -h on the scripts that might be a bit long since hostchk.sh is run by default via make. But perhaps the version alone is enough for this part?

xexyl commented 2 years ago

Do you think the user needs to see the output of all these commands? If not and that's actually the problem it would greatly simplify the problem. But like I said I've not really looked into it. What are your thoughts?

Probably not. One could run iocccsize_test.sh directly, for example, with a -v level if they wanted more information.

Or in this case they can use -v in the bug_report.sh script too. Thus if they specify a verbosity level (say greater than 1 - not sure if it should be > 0 or > 1 but I'll take a look at what else exists verbosity wise .. or maybe it could be a debug level even) it would show this information but otherwise it would not. That might be the way to go. By default it can be quieter but if they want more information they can then specify -v 1 or whatever.

xexyl commented 2 years ago

I thought I saw this yesterday but there seems to be an issue with the hostchk_warning rule in the Makefile. Not an issue with the rule itself so much that it's kind of a circular (but not fully) dependency. For instance if one runs:

./bug_report.sh

and hostchk.sh exits 1 you'll see the warning more than once. This is because, it seems, that all runs hostchk_fast which runs hostchk.sh as well. But then for bug_report.sh we want to run it with more detailed information so should all actually run it? I'm not sure. I think it would be confusing to the user although perhaps if it fails it's not a bad thing if it warns twice.

But there's another part to it. The warning says to use bug_report.sh which also uses hostchk.sh so that warning will be shown again. Is this overkill? Is it the right amount? I'm mixed on this.

It might be good in that it'll really stress that there's an issue. But it might be too much for some people because the bug_report.sh actually uses hostchk.sh which if it exits non zero it says to use bug_report.sh again. I seriously doubt any IOCCC contestant will be confused with this but it might be something that some might consider to be an issue. On the other hand it would be a way that allows for you and Leo (and me in this case as I'm writing the scripts) to say: we gave you plenty of warnings so you can't say that we didn't warn you that the toolkit might have problems!

With that in mind I think I'll keep it that way but I'll make a comment in the scripts about this. If you feel differently though feel free to edit the scripts or Makefile rule (maybe depending on the option like you do for run_bison.sh and run_flex.sh ?).

xexyl commented 2 years ago

One issue that I noticed yesterday that might be confusing:

If ioccc_test.sh passes all tests then it will report a message like:

All tests PASSED

But then if something else fails after that - say hostchk.sh it will report that there is an issue right then. I have some ideas of what can be done but I must leave for now. Will try updating this post later on. Either way it should be addressed.

xexyl commented 2 years ago

See commits 79cef044d6f50601a25eb30a9c4e6fabfc6af4bf and a77d781f01f2762c19ddcbb208285f4932dcaacb. There are some amusing comments in there for you and others.

For instance about the bug that was fixed:

# The below references to PIPESTATUS (instead of $?) fix the bug introduced in commit
# 8343c4b8cb97e52df64fe8973e68f0d83c6090e1 where the exit status of each command
# was not checked properly which meant that even if a test failed it would not
# be reported as an issue which rather defeated the purpose of this script.
#
# As amusing as the thought is that there's a bug in a script to help report
# bugs and issues, this bug was not in fact intentional. :-) I had thought of it
# earlier on but I neglected to address it before the commit as I moved on with
# adding the tests. However since the world definitely needs more reasons to
# laugh I'm keeping this comment here for the sake of humour and the irony that
# I caused. You're welcome! :-)

There's also the discussion about the loop of scripts:

# NOTE: This will indirectly call make fast_hostchk which, if it reports an
# issue, will be reported here. However we also directly invoke hostchk.sh later
# which will also report an issue so that one would likely see the warning more
# than once. If you only see the warning once then there probably is also a
# problem. The warning issued will suggest that you can file the issue with
# this script and it also suggests that you can get more information with
# running hostchk.sh directly.
#
# Obviously one needn't run this script a second or third time just because it
# runs hostchk.sh which suggests that you run this script each time it exits
# non-zero! :-) Sorry in advance if this (to make use of an American English
# phrase which I will happily use for a pun for a pun not made is a wasted
# opportunity :-) ) throws you for a loop! :-) But now that you're in the loop
# it shouldn't even matter. :-)

A similar note is in hostchk.sh.

I also created a man page for both scripts. Since I need to go afk now I will not (at least for now) update the other threads with this. As it is I'm not even sure which issues I said I would update my comments. Anyway I'm not sure if I'll do anything else here today as I'm quite tired but I do feel good about what I got done. Afk now.

xexyl commented 2 years ago

Again the purpose of this make bug-report rule is to create a bug-report.$(/bin/date '+%%Y%m%d.%H%M%S').txt file (out a timestamp containing information information that might be helpful to understand why someone might be having a problem compiling and running code.

Use the output of $(/bin/date '+%%Y%m%d.%H%M%S') to put a timestamp into the filename.

So things like the following could be written to this file:

  • make clobber
  • uname -a [...]

I thought of a problem with uname -a: it will show the host name or even FQDN. This does not bother me but it might bother other people. So what options should be used? I propose:

uname -srvm

which works under both linux and macOS. The options in linux -oi are redundant and macOS does not have them.

xexyl commented 2 years ago

Again the purpose of this make bug-report rule is to create a bug-report.$(/bin/date '+%%Y%m%d.%H%M%S').txt file (out a timestamp containing information information that might be helpful to understand why someone might be having a problem compiling and running code.

Use the output of $(/bin/date '+%%Y%m%d.%H%M%S') to put a timestamp into the filename.

So things like the following could be written to this file:

  • make clobber
  • uname -a
  • cc -v
  • which cc
  • which tar
  • if tar supports the required flags
  • make all
  • ./run_bison.sh -v 1
  • ./run_flex.sh -v 1
  • ./hostchk.sh -v 1
  • make test

As far as tar: I don't think we need to worry about it since it calls hostchk.sh which tries to use the correct options and if it fails it will be reported. We might want to collect the path to tar though. Some of the others are already done as of yesterday. Others can be done in coming updates.

xexyl commented 2 years ago

From comment https://github.com/ioccc-src/mkiocccentry/issues/248#issuecomment-1153283608 (which I need to refer to later at another time - not today most likely as I think I'm done with the repo today - as it has lots of good stuff that will be helpful to add):

Then each of the executables should be tested if they exist and are executable. If so then run them with the -h flag:

  • ./dbg -h
  • ./dyn_test -h
  • ./fnamchk -h
  • ./hostchk.sh -h
  • ./ioccc_test.sh -h
  • ./iocccsize -h
  • ./iocccsize_test.sh -h
  • ./jauthchk -h
  • ./jcodechk.sh -h
  • ./jinfochk -h
  • ./jnum_chk -h
  • ./jnum_gen -h
  • ./jparse -h
  • ./jparse_test.sh -h
  • ./json_test.sh -h
  • ./jstr_test.sh -h
  • ./jstrdecode -h
  • ./jstrencode -h
  • ./mkiocccentry -h
  • ./mkiocccentry_test.sh -h
  • ./prep.sh -h
  • ./reset_tstamp.sh -h
  • ./run_bison.sh -h
  • ./run_flex.sh -h
  • ./txzchk -h
  • ./txzchk_test.sh -h
  • ./utf8_test -h
  • ./verge -h
  • ./vermod.sh -h

Do we need to explicitly check for it being executable? My thinking is if it's not we'll find out when we try getting help. Of course we could say that if we check for it being executable we could then skip any tests with it but that would require either a large array or a lot of variables (at least I fit's not a simple matter of one if else block). What do you think ?

An argument for first checking is it might look cleaner; an argument for not checking is it'll take more code for information that can possibly be inferred indirectly. An argument for it is that it will be clearer what is wrong if we do check. It might also be easy enough to duplicate in vim so that we don't have to write much (more yank paste and substitute). It's just a thought that popped into my head. I guess it probably is better to explicitly check especially since if it's not executable it would narrow the problem down.

On the other hand if we looked at the output and saw that it's not executable we would also know so that that test would be inferred indirectly. For example:

$ ./txzchk_test.sh
-bash: ./txzchk_test.sh: Permission denied

but that could be not so clear either. Still since it popped into my head I'll keep the thought here.

If the -h for a given code does NOT include a version string, it should. That way we have a better idea of what version their code is.

I am working on this. I think that in the near future I will verify that they all have versions and those that don't I will add them.

.. and anything else you might think might be useful to try and understand the user's problematic environment.

The Vogons must be the root of their problems! :-)

lcn2 commented 2 years ago

This makes me think that maybe there ought to be several more files. What do you think about that? I'm kind of thinking along the lines of test_iocccsize/. Should we have a directory for hostchk too? I'm starting to think that would be a good idea.

That might be an idea to consider.

Thank you for confirming. I'll look into it later on - tomorrow or in the coming days. The compiling with all the needed header files is good the way it's being done because it means if any header file is changed (new one needed or one no longer needed) it'll be picked up. I know there are some other comments in this thread too that I guess hostchk should use. Is that right? I mean should the compiler version and flex/bison versions and make etc. all be in hostchk ? Or should it be in bug report ? I refer btw to this comment (for example): #248 (comment).

Well the bug-report.sh tool should produce info about flex/bison (path, version) if they exist. The hostchk.sh doesn't need to check flex/bison because the Makefile and run_bison.sh and run_flex.sh mange that already.

Sure but the question is should bug_report.sh check the version of the other tools .. or should it let hostchk.sh do that? Given that you say hostchk doesn't need to because the Makefile does I guess it means that if the Makefile had not (via the two scripts) it would have been the hostchk.sh script.

The bug_report.sh should just print versions, documenting what the user has. No need to check values.

There isn't anything that hostchk.sh needs to check in terms of tool versions.

lcn2 commented 2 years ago

To clarify why I think first of hostchk.sh: it's because it does a check on the host and checking the version on the host seems like a task for such a script. If however it's supposed to run -h on the scripts that might be a bit long since hostchk.sh is run by default via make. But perhaps the version alone is enough for this part?

Given comment 1279578628, the hostchk.sh might only need to check if shell scripts used in the process of compiling, exist and are executable.

As most shell scripts are NOT used for compiling, most shell scripts can be ignored by hostchk.sh.

IMPORTANT POINT

Remember that the purpose of hostchk.sh is to quickly determine of the environment is likely to be problematic in compiling the repo.

Also: hostchk.sh -f should be fast, silent if nothing is wrong, as this code impacts the time to compile the repo.

lcn2 commented 2 years ago

I thought I saw this yesterday but there seems to be an issue with the hostchk_warning rule in the Makefile. Not an issue with the rule itself so much that it's kind of a circular (but not fully) dependency. For instance if one runs:

./bug_report.sh

and hostchk.sh exits 1 you'll see the warning more than once. This is because, it seems, that all runs hostchk_fast which runs hostchk.sh as well. But then for bug_report.sh we want to run it with more detailed information so should all actually run it? I'm not sure. I think it would be confusing to the user although perhaps if it fails it's not a bad thing if it warns twice.

But there's another part to it. The warning says to use bug_report.sh which also uses hostchk.sh so that warning will be shown again. Is this overkill? Is it the right amount? I'm mixed on this.

It might be good in that it'll really stress that there's an issue. But it might be too much for some people because the bug_report.sh actually uses hostchk.sh which if it exits non zero it says to use bug_report.sh again. I seriously doubt any IOCCC contestant will be confused with this but it might be something that some might consider to be an issue. On the other hand it would be a way that allows for you and Leo (and me in this case as I'm writing the scripts) to say: we gave you plenty of warnings so you can't say that we didn't warn you that the toolkit might have problems!

With that in mind I think I'll keep it that way but I'll make a comment in the scripts about this. If you feel differently though feel free to edit the scripts or Makefile rule (maybe depending on the option like you do for run_bison.sh and run_flex.sh ?).

The make hostchk_warning does not run anything, only prints to stderr and pauses. This is for a very good reason, among which is the circular problems you hint at above. Thus make hostchk_warning does what is needs to do and nothing more.

xexyl commented 2 years ago

To clarify why I think first of hostchk.sh: it's because it does a check on the host and checking the version on the host seems like a task for such a script. If however it's supposed to run -h on the scripts that might be a bit long since hostchk.sh is run by default via make. But perhaps the version alone is enough for this part?

Given comment 1279578628, the hostchk.sh might only need to check if shell scripts used in the process of compiling, exist and are executable.

As most shell scripts are NOT used for compiling, most shell scripts can be ignored by hostchk.sh.

Remember that the purpose of hostchk.sh is to quickly determine of the environment is likely to be problematic in compiling the repo.

Still here briefly.

Just to be sure: the bug_report script does need to make the check on all these scripts though, right? Meanwhile the other script just needs to be extended to compile those examples you included the other day so it doesn’t need to check all the scripts (that they are executable).

xexyl commented 2 years ago

I thought I saw this yesterday but there seems to be an issue with the hostchk_warning rule in the Makefile. Not an issue with the rule itself so much that it's kind of a circular (but not fully) dependency. For instance if one runs:

./bug_report.sh

and hostchk.sh exits 1 you'll see the warning more than once. This is because, it seems, that all runs hostchk_fast which runs hostchk.sh as well. But then for bug_report.sh we want to run it with more detailed information so should all actually run it? I'm not sure. I think it would be confusing to the user although perhaps if it fails it's not a bad thing if it warns twice. But there's another part to it. The warning says to use bug_report.sh which also uses hostchk.sh so that warning will be shown again. Is this overkill? Is it the right amount? I'm mixed on this. It might be good in that it'll really stress that there's an issue. But it might be too much for some people because the bug_report.sh actually uses hostchk.sh which if it exits non zero it says to use bug_report.sh again. I seriously doubt any IOCCC contestant will be confused with this but it might be something that some might consider to be an issue. On the other hand it would be a way that allows for you and Leo (and me in this case as I'm writing the scripts) to say: we gave you plenty of warnings so you can't say that we didn't warn you that the toolkit might have problems! With that in mind I think I'll keep it that way but I'll make a comment in the scripts about this. If you feel differently though feel free to edit the scripts or Makefile rule (maybe depending on the option like you do for run_bison.sh and run_flex.sh ?).

The make hostchk_warning does not run anything, only prints to stderr and pauses. This is for a very good reason, among which is the circular problems you hint at above. Thus make hostchk_warning does what is needs to do and nothing more.

Yes. Agreed. I think it’s a good idea that it ends up showing up more than once as it really will stress that there’s a problem. This way they can’t say that they weren’t warned.

Okay and now I am going to put the phone down. Hopefully I have replied enough so that you can make any additional comments that will be enough for me to continue tomorrow. Hopefully sleep will be at least okish for me and I can get more done.

I feel good about what I did today. I think that besides the documentation and the json stuff (like chkentry) the only other thing that need to be done (checking for consistency and typos etc.) is these two scripts. Am I right?

Well anyway if nothing else I can continue dialogue tomorrow. Good night my friend!

lcn2 commented 2 years ago

One issue that I noticed yesterday that might be confusing:

If ioccc_test.sh passes all tests then it will report a message like:

All tests PASSED

But then if something else fails after that - say hostchk.sh it will report that there is an issue right then. I have some ideas of what can be done but I must leave for now. Will try updating this post later on. Either way it should be addressed.

In general, and part of the general Unix tool philosophy, tests shouldn't wrote unnecessary output as this complicates using it as input to another tool.

See commit 29c791b837bcc2d66c4b0a42825f66211dfd3f68

lcn2 commented 2 years ago

I also created a man page for both scripts. Since I need to go afk now I will not (at least for now) update the other threads with this. As it is I'm not even sure which issues I said I would update my comments. Anyway I'm not sure if I'll do anything else here today as I'm quite tired but I do feel good about what I got done. Afk now.

Thanks and best wishes for a good rest.

xexyl commented 2 years ago

One issue that I noticed yesterday that might be confusing: If ioccc_test.sh passes all tests then it will report a message like:

All tests PASSED

But then if something else fails after that - say hostchk.sh it will report that there is an issue right then. I have some ideas of what can be done but I must leave for now. Will try updating this post later on. Either way it should be addressed.

In general, and part of the general Unix tool philosophy, tests shouldn't wrote unnecessary output as this complicates using it as input to another tool.

See commit 29c791b

Yes. And thank you.

xexyl commented 2 years ago

I also created a man page for both scripts. Since I need to go afk now I will not (at least for now) update the other threads with this. As it is I'm not even sure which issues I said I would update my comments. Anyway I'm not sure if I'll do anything else here today as I'm quite tired but I do feel good about what I got done. Afk now.

Thanks and best wishes for a good rest.

Yes and you too!

I was just looking at some funny German cartoons but now I am going (no I am not fluent but I know some German.)

Sleep well my friend! More from me tomorrow I am sure! Good night!

lcn2 commented 2 years ago

Again the purpose of this make bug-report rule is to create a bug-report.$(/bin/date '+%%Y%m%d.%H%M%S').txt file (out a timestamp containing information information that might be helpful to understand why someone might be having a problem compiling and running code.

Use the output of $(/bin/date '+%%Y%m%d.%H%M%S') to put a timestamp into the filename.

So things like the following could be written to this file:

  • make clobber
  • uname -a [...]

I thought of a problem with uname -a: it will show the host name or even FQDN. This does not bother me but it might bother other people. So what options should be used? I propose:

uname -srvm

which works under both linux and macOS. The options in linux -oi are redundant and macOS does not have them.

Run uname multiple times ... do both uname -a and uname -srvm.

lcn2 commented 2 years ago

Do we need to explicitly check for it being executable? My thinking is if it's not we'll find out when we try getting help. Of course we could say that if we check for it being executable we could then skip any tests with it but that would require either a large array or a lot of variables (at least I fit's not a simple matter of one if else block). What do you think ?

Check for the existence of the prog, if it is a readable file, if it is executable, and then record the output of -h.

UPDATE 0

The purpose of bug-report.sh is to try and collect in the bug report, as much information as we can, that might be useful in debugging a problem (such as cannot build the repo, or stuff doesn't execute, etc.).

xexyl commented 2 years ago

Again the purpose of this make bug-report rule is to create a bug-report.$(/bin/date '+%%Y%m%d.%H%M%S').txt file (out a timestamp containing information information that might be helpful to understand why someone might be having a problem compiling and running code.

Use the output of $(/bin/date '+%%Y%m%d.%H%M%S') to put a timestamp into the filename.

So things like the following could be written to this file:

  • make clobber
  • uname -a [...]

I thought of a problem with uname -a: it will show the host name or even FQDN. This does not bother me but it might bother other people. So what options should be used? I propose:

uname -srvm

which works under both linux and macOS. The options in linux -oi are redundant and macOS does not have them.

Run uname multiple times ... do both uname -a and uname -srvm.

But the problem is that it will show host name which some people might not like? And why run it twice?

Well anyway that can be decided in the coming days.

I am starting to sit up so that I will be forced to get ready to sleep. Take good care and I will reply more tomorrow.

lcn2 commented 2 years ago

Again the purpose of this make bug-report rule is to create a bug-report.$(/bin/date '+%%Y%m%d.%H%M%S').txt file (out a timestamp containing information information that might be helpful to understand why someone might be having a problem compiling and running code.

Use the output of $(/bin/date '+%%Y%m%d.%H%M%S') to put a timestamp into the filename.

So things like the following could be written to this file:

  • make clobber
  • uname -a [...]

I thought of a problem with uname -a: it will show the host name or even FQDN. This does not bother me but it might bother other people. So what options should be used? I propose:

uname -srvm

which works under both linux and macOS. The options in linux -oi are redundant and macOS does not have them.

Run uname multiple times ... do both uname -a and uname -srvm.

But the problem is that it will show host name which some people might not like? And why run it twice?

Well anyway that can be decided in the coming days.

I am starting to sit up so that I will be forced to get ready to sleep. Take good care and I will reply more tomorrow.

The bug report log is only generated. People are free to edit it as they like, or even not sent it.

Collect information into the bug report. Tell the user that they can read the bug report in the file that was created.

No need to handle the case where someone might be offended by what is produced in the bug report file.

xexyl commented 2 years ago

Good morning! I hope you're sleeping well. I did get a few commits in but I'm not feeling so well now. I'm going to reply to comments and then see how I feel. I might or might not get more done after this. If I don't go to work on it right away I might do it later on.

Do we need to explicitly check for it being executable? My thinking is if it's not we'll find out when we try getting help. Of course we could say that if we check for it being executable we could then skip any tests with it but that would require either a large array or a lot of variables (at least I fit's not a simple matter of one if else block). What do you think ?

Check for the existence of the prog, if it is a readable file, if it is executable, and then record the output of -h.

How ironic that a commit I accidentally made yesterday removed the execute bit from hostchk.sh when we're talking about executable scripts!

Will do these checks but do we need the full output of the help mode? Or will version be enough? I'm just thinking out loud here:

On the one hand the version should tell us what we need to know but on the other hand if there's some kind of modification they made it might be hard to tell based on that output as it'll only show the version. At the same time if one is to modify the file in some way (or it's modified because of faulty RAM ... in which case we probably can't trust anything else from them either but never mind that here ...) we won't know unless so is this necessary? Still save for having them send everything to us (which would be too much I think) this is a good idea though not shown in output to them. Perhaps the next thing I need to do is limit the output shown at the console and write it mostly to the log file only.

And of course if they did modify the script or anything else we'll find out (most likely) when discussing it with them. So I guess it is that we will have the help string printed in full if it's executable. If it's not it'll be reported as such.

UPDATE 0

The purpose of bug-report.sh is to try and collect in the bug report, as much information as we can, that might be useful in debugging a problem (such as cannot build the repo, or stuff doesn't execute, etc.).

Of course.