Enhancement: Prepare for a formal release

[lcn2]

Before the mkiocccentry repo has a formal release, certain things should be performed. This request an action item list to serve as a reminder before this is done.

TODO:

• [x] Establish a overall repo release version style to be used as a git tag IOCCC_TOOLSET_RELEASE in limit_ioccc.sh
• [x] Establish a formal release tag commit string Release x.y for IOCCC entry tool set
• [x] Create a command line tool to print the repo release version string ./mkiocccentry -T
• [x] Create a "what is new in this release of the mkiocccentry tool set" change document See CHANGES.md
• [x] Add a bunch of test cases for jauthchk under test_JSON/author.json While test_JSON/info.json has a number of cases, test_JSON/author.json does not
• [x] Add any missing jinfochk test cases under test_JSON/info.json Any new test stations that may have come up recently should be added, if any are needed
• [x] Add tests for jauthchk to make test Perhaps this should be done via a new combined json-test.sh script that make test can call?
• [x] Add tests for jinfochk to make test Perhaps this should be done via a new combined json-test.sh script that make test can call?
• [x] Make is possible for someone without flex and bison to compile the repo
• [x] Change 1st, 2nd etc. to words and change '2 args' to 'two args', and so on
• [x] Modify txzchk to be able to read a file as it it were the output of some tar -t... command Build a directory of both good and bad *.txt files (instead of good and bad tarballs` as test cases.
• [x] Complete and close #156
• [x] Complete and close #68 Us the JSON parse to examine the JSON parse tree to verify that the parsed JSON is appropriate for an .info.json file.
• [x] Complete and close #69 Use the JSON parser to examine the JSON parse tree to verify that the parsed JSON is appropriate for an .author.json file.
• [x] Complete and close #259 Use the JSON semantic checks as JSON parser to verify .info.json and author.json files w/o spoiling authorship.
• [x] Complete and close #239 Replace binary data blobs under test_JSON with c blobs.
• [x] Complete and close #248 Add a make bug-report that would run the test suite and write various info to say bug-report.txt.
• [x] Complete and close #250 The make hostchk rule performs some basic checks to see if the host can support this repo. If a problem is detected, a suggestion could be given to the user. For example, if they are running under an OS that does not have a modern enough tar (such as perhaps IRIX or maybe an old OpenBSD), one should recommend they download a install a more up to date tar such as GNU Tar.

Release 0.9.9 actions: (once all the above TODO actions are done)

• [x] Setup a Makefile sub-directory mechanism

• [x] Change from asking for optional twitter handles to optional Mastodon usernames

• [x] Move dbg code into its own sub-directory

• [x] Move dyn_array code into its own sub-directory

• [x] Move JSON parser code into its own sub-directory

• [x] Sync dbg code with the dbg repo

• [x] Revise or improve the repo discussion or remove it

• [x] Add info on how to report a bug, look at diagnostics, etc.

• [x] Add info on how a IOCCC user can build what they need to submit an entry to the IOCCC

• [x] Resolve all XXX commented issues

• [x] Check if older versions of flex and/or bison will work Adjust the minimum version strings accordingly. NOTE: bison 2.3 does NOT work.

• [x] Make sure that we haven't disabled any warnings that matter Make sure that disabled warnings are truly needed and don’t mask any problems.

• [x] Complete issue #448

• [x] Complete issue #449

• [x] Complete issue #453

• [x] Complete issue #458

• [x] Complete issue #459

• [x] Complete issue #461

• [x] Complete issue #475

• [x] Complete issue #484

• [x] Perform static analysis Consider topics of analysis such as those in The state of static analysis in the GCC or in clang under MacOS. Also look for major memory leaks. Create a test_ioccc/static_analysis.md file as a mini-tutorial on how to perform static analysis, and list the classes of warnings produced that are being deliberately ignored and why.

• [x] Review what is included in *.[chly] files Remove any unnecessary includes, if needed. However there is not need to go to an extreme minimum, just remove any include that are not strictly needed. Limit includes under dbg/ to only system and dbg related files. Limit includes under dyn_alloc/ to only system, _dynalloc, and dbg related files. Limit includes under jparse/ to only system, jparse, _dynalloc, and dbg related files.

• [x] Review the list of *.o files for each executable made by this repo Remove any use of *.o that is not needed for a given specific binary. As much of the underlying code is being added via static libraries, they may be nothing needed for this checklist item.

• [x] Adjust dbg() and json_dbg() debugging levels Adjust the debug trigger levels so that more verbose detailed debugging only prints at higher levels of debugging: ONLY if the current debugging verbosity is annoying. There may be nothing that needs to be done with this checklist item.

• [x] Verify that make test does not depend on rules that make other things The test rule should not depend on all, for example.

• [x] Verify in C code, exit codes explicitly mentioned in man pages use the correct exit code and use /*ooo*/ to prevent seqcexit(1) from charing them Adjust man pages and/ or C code as needed while maintaining the exit codes that are already documented

• [x] Review all things in the code marked as "TODO" or "XXX" Any TODO or XXX that can reasonably be done, and MUST be done before public review, should be completed.

• [x] Review all error messages - correct / improve only if needed

• [x] Review README.md Review from the perspective of the general IOCCC interested public who has just been invited to review this repo.

• [x] go through all files to check for consistency problems and typos

• [x] Review all documentation: update documentation as needed

• [x] Update CHANGES.md to include all important/significant post version 0.9.9 release work Be sure that all major/important changes made to the code and documentation,since Release 0.9.9, are listed.

• [x] have a final freeze pass at foo related fun prior to the version 1.0.0 code freeze

• [x] Briefly review all of the previously checked (completed) checklist items Briefly verify, where reasonable, that the item was done or is no longer needed. It is possible that a stray finger could have improperly closed an item before it was completed. Reopen any such items that really need to be completed and perform them. There may be nothing that needs to be done with this checklist item.

Code complete actions: (once all above Release 0.9.9 actions done)

• [x] Update to version 1.0.0 and CHANGS.md
• [x] Change versions of tools that are < 1.0 to 1.0.0
• [x] Clean out legacy_clean and legacy_clobber actions
• [x] Update dbg repo to be equivalent to files under dbg/
• [x] make picky Resolve all picky issues
• [x] make shellcheck Resolve all shellcheck issues
• [x] Remove any trailing blanks from lines i.e., look for: /\s+$/
• [x] Remove any improper spaces between leading tabs i.e., look for: /^[\t ]* \t/
• [x] Remove the -Werror flag on WARN_FLAGS
• [x] Change C_OPT to use -O3 -g3 ready
• [x] run ./soup/reset_tstamp.sh Make note of the previous and current minimum timestamp
• [x] Update timestamps under test_JSON as needed Do not change the test cases where the timestamp we intentionally set to be early
• [x] Update CHANGES.md as needed
• [x] Push all pending changes

Final test: (once Code complete actions is done) - try on many reasonably up to date platforms:

• [x] git stash list Verify nothing is stashed. If any stashed files are found, remove stashes and restart this section from the beginning.
• [x] inspect make clobber ; git status --ignored Verify that the correct files are being excluded. If any unusual files are being ignored, resolve the issue and restart this section from the beginning.
• [x] git status Verify that the directory tree is now clean. If the directory tree is not clean, resolve the issue and restart this section from the beginning.
• [x] make release or make prep as needed Verify that the make action is OK. If any problems are found, fix them and restart this section from the beginning.
• [x] make all Verify that the make all does nothing and prints nothing after doing make release. If make all does something and/or prints something, fix the issue and restart this section from the beginning.
• [x] git status Re-verify that the directory is now clean. If the directory tree is not clean, resolve the issue and restart this section from the beginning.
• [x] inspect make clobber all ; make clobber ; git status Verify the directory is now clean after make clobber. If the directory tree is not clean, resolve the issue and restart this section from the beginning.
• [x] inspect git status --ignored Verify that the correct files are being excluded. If any unusual files are being ignored, resolve the issue and restart this section from the beginning.
• [x] inspect make clobber all test Verify that the make action is OK. If any problems are found, fix them and restart this section from the beginning.

Formal release 1.0.0 (once Final test is done)

• [x] Complete and close #171
• [x] Sync dbg/ into the dbg repo
• [x] Form a formal release git commit using the formal release tag commit string NOTE: Add the recent what is new in this release below the command tag title
• [x] push formal release git commit 1st commit line: release mkiocccentry repo version X.Y.Z 2nd commit line: 3rd commit line: Version X.Y.Z of mkiocccentry repo.
• [x] set release version: VERSION=X.Y.Z
• [x] set repo release version: REPO_VERSION="v$VERSION"
• [x] set formal release tag commit string: REPO_COMMIT="version $VERSION of mkiocccentry repo"
• [x] Add a git tag git tag "$REPO_VERSION" -m"$REPO_COMMIT"
• [x] Verify tag git tag ; git describe --tags --abbrev=0
• [x] push tags git push --tags git@github.com:ioccc-src/mkiocccentry.git master
• [x] go to the tags screen https://github.com/ioccc-src/mkiocccentry/tags
• [x] click on the latest tag
• [x] click ((Create release from tag))
• [x] enter release title Set release title to be the output of: echo "mkiocccentry $REPO_VERSION release"
• [x] Content: Enter the '## Release' line and the rest of the section text from CHANGES.md
• [x] examine Content by clicking ((Preview)) Edit if and as needed
• [x] Select [x] Create a discussion for this release
• [x] click ((Publish release))
• [x] Complete and close #87
• [x] git fetch && git pull && git clean -f && git describe --tags --abbrev=0
• [ ] Announce the 1st release of mkiocccentry
• [ ] Ask for public feedback on the releases

[xexyl]

Just to be sure: you have it in issues because you want feedback, right?

As for:

Remove any trailing blanks from lines
i.e., look for: /\s+$/

I often do this. I have it visible in git diff (I'm not actually sure if this is a default but I think it might be because of coloured diff and the fact that a single space can make a diff). I actually have commented out in my .vimrc a way to do this when saving files but I have it commented out exactly because of the git diff being annoying (and not wanting to make it much of the commit).

Anyway this is how you can do it in .vimrc:

        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e

That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

let c_space_errors = 1

Perhaps it highlights spaces? I'm not sure.

Are you thinking of changing the name of the repo btw? I wasn't sure from the first part of your post. And now I'll be leaving for the day.

EDIT 0, 12 January 2023

Typo fixes.

[lcn2]

Just to be sure: you have it in issues because you want feedback, right?

Yes.

[lcn2]

Are you thinking of changing the name of the repo btw? I wasn't sure from the first part of your post. And now I'll be leaving for the day.

Should the repo name change? If so, to what and why?

[xexyl]

Are you thinking of changing the name of the repo btw? I wasn't sure from the first part of your post. And now I'll be leaving for the day.

Should the repo name change? If so, to what and why?

I don't think so, no. I just got that impression that you might be thinking of it - from the first part of your post. It was a very quick glance.

[xexyl]

Just to be sure: you have it in issues because you want feedback, right?

As for:

Remove any trailing blanks from lines
i.e., look for: /\s+$/

I often do this. I have it visible in git diff (I'm not actually sure if this is a default but I think it might be because of coloured diff and the fact that a single space can make a diff). I actually have commented out in my .vimrc a way to do this when saving files but I have it commend out exactly because of the git diff being annoying (and not wanting to make it much of the commit).

Anyway this is how you can do it in .vimrc:

        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e

That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

Of course we could also use sed to do a single pass on this with the -i option if it just needs to be done at the end but maybe the vim config is useful too.

And now I really am going. Have a great afternoon! I'll read more in the other thread tomorrow (or maybe later today and react tomorrow).

Cheers.

[lcn2]

Anyway this is how you can do it in .vimrc:

        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e

That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

Nice hack!

let c_space_errors = 1

Perhaps it highlights spaces? I'm not sure.

The vim help says:

*c_space_errors*                trailing white space and spaces before a <Tab>

But it does not seem to do anything.

[xexyl]

Anyway this is how you can do it in .vimrc:

        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e

That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

Nice hack!

Yes. And I remembered that although I have it disabled I also added to my .vimrc that it attempts to read in from the current directory the file .override.vim which means I could enable it for this repo. Want me to do that?

let c_space_errors = 1

Perhaps it highlights spaces? I'm not sure.

The vim help says:

*c_space_errors*                trailing white space and spaces before a <Tab>

But it does not seem to do anything.

Yes I looked at that too. I don’t know what it does and it’s possible that that’s why I have it disabled. But I have this vague memory that I disabled it due to not liking the effect it had. But what that effect is I have not the slightest clue.

I will reply to the other things you wrote in the morning. I hope you have a great night! If you have any thoughts on the other things I mentioned I would be very interested in them but it sounds like you have quite some changes you’re making so perhaps that’s why you haven’t got to it yet.

Well no rush on my behalf. I will add what I can do and the problems can be worked out later.

Good night!

EDIT on 13 April 2022: typo fix. Bloody phone typing.

[xexyl]

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

If there are any problems I could probably fix them. Actually years ago he had me port linux_logo to BSD. I was going to help him get some things working on this computer but it’s now rather impossible for me to do that since ssh is disabled. But I probably can get him to test this repo.

There are some other things in the list of things to do that I can help with. Documentation included. I should make man pages for the recent tools - the decode and encode tools. I just have been focused on the ones that haven’t been completed yet. But perhaps the next time I need a break (not from being too tired) I can do that. We shall see.

And with that I am going to put the phone down. Expect a pull request tomorrow!

[lcn2]

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

FYI: Landon worked at SGI for a number of years.

[xexyl]

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

[xexyl]

Anyway this is how you can do it in .vimrc:

        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e

That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

Nice hack!

Yes. And I remembered that although I have it disabled I also added to my .vimrc that it attempts to read in from the current directory the file .ovveride.vim which means I could enable it for this repo. Want me to do that?

let c_space_errors = 1

Perhaps it highlights spaces? I'm not sure.

I was right about what that option does - it highlights trailing spaces. See the screenshot .

Edit: Which of course has a comment that's not true, perhaps from yank and paste. I'll fix it.

[lcn2]

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

I have an SGI O2, but I don't power it up these days. Silicon Graphics was a fun company in its day.

[xexyl]

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

I have an SGI O2, but I don't power it up these days. Silicon Graphics was a fun company in its day.

I bet it was fun! My mum actually worked for the telco and she got to use the original AT&T Unix. I was always jealous of that. She also one time got to use a punchcard (which I was also jealous of) - but that was on strike duty.

I asked my friend what model - hopefully he'll get back to me sometime in the near future.

[xexyl]

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

I have an SGI O2, but I don't power it up these days. Silicon Graphics was a fun company in its day.

That's the same model he has.

[xexyl]

About to go to sleep but I just by chance saw this. I am too tired to even read it but it might be something we can use at some point:

https://developers.redhat.com/articles/2022/04/12/state-static-analysis-gcc-12-compiler

[xexyl]

I thought of two other things to do. One I'll do soon most likely; the other can probably come later.

The first one is change 1st, 2nd etc. to words. Similarly '2 args' should be 'two args' and so on. This should be easy enough to find most cases (the first part very easy but the second part might take a bit more time because [1-9]+ has some things that shouldn't be changed. Two examples:

In util.c:

util.c:     * must be a UUID.  The UUID has the 36 character format:
util.c:     *       xxxxxxxx-xxxx-4xxx-axxx-xxxxxxxxxxxx
util.c:     * where 'x' is a hex character.  The 4 is the UUID version and the variant
util.c:     * 1.

The 36 is correct and the 4 and 1 should stay the same as well and there might be other places where this applies (and anything > 9 should be digits except maybe for one hundred and the like: I suppose that this is up to debate on the latter part though). Another example that can't be updated is the UTF-8 map.

Of course I could change the regex to be just [0-9] or maybe [1-9] but you get the idea: it's not always clear cut.

But actually all of it might be up to debate in some minds except I think the 1st -> first and similar should be done and for args two args etc. The rest can be decided but then it should also be consistent which I guess is the most important thing.

The second thing is we should give a bit more help when tools are missing that can be downloaded somewhere. In particular we give a download link to tar but only for linux. It might be an idea to explain how to do it for macOS for example. I guess it's more extreme to check which system is being used and print the correct links/help but it did cross my mind so figured I'd mention it anyway.

Something else just occurred to me: does anyone have any big endian machines to test everything under?

[lcn2]

The first one is change 1st, 2nd etc. to words. Similarly '2 args' should be 'two args' and so on ...

This item has been added to the TODO list above.

[xexyl]

The first one is change 1st, 2nd etc. to words. Similarly '2 args' should be 'two args' and so on ...

This item has been added to the TODO list above.

I can do the that one fairly easily but I don't want to stomp on any of your pending changes so I'm waiting on that. Hoping the pull request that I did today does not but as I said I can redo the JSON renaming again.

[lcn2]

The second thing is we should give a bit more help when tools are missing that can be downloaded somewhere. In particular we give a download link to tar but only for linux. It might be an idea to explain how to do it for macOS for example. I guess it's more extreme to check which system is being used and print the correct links/help but it did cross my mind so figured I'd mention it anyway.

That does seem a bit extreme. For the make all that people will need to use, the minimum is the C compiler and common command line tools. For those with systems that lack something like tar, we mention where to get it.

That is probably as far as we should go. Downloading software should be the responsibility of the user. it may be too far to download and try to install something.

In the worst case, the user simply needs to find a system that as common command line tools. This repo does NOT have to be compiled on the system for which the entry was built. Users can copy the source for their entry to a system that as common command line tools and compile / use this repo there.

[xexyl]

The second thing is we should give a bit more help when tools are missing that can be downloaded somewhere. In particular we give a download link to tar but only for linux. It might be an idea to explain how to do it for macOS for example. I guess it's more extreme to check which system is being used and print the correct links/help but it did cross my mind so figured I'd mention it anyway.

That does seem a bit extreme. For the make all that people will need to use, the minimum is the C compiler and common command line tools. For those with systems that lack something like tar, we mention where to get it.

That is probably as far as we should go. Downloading software should be the responsibility of the user. it may be too far to download and try to install something.

In the worst case, the user simply needs to find a system that as common command line tools. This repo does NOT have to be compiled on the system for which the entry was built. Users can copy the source for their entry to a system that as common command line tools and compile / use this repo there.

Well I'd agree that the last part is extreme. I am not sure if giving them a link for macOS is though. On the other hand macOS has tar built in as well so it's probably fine.

I wasn't suggesting downloading or installing though! That would be very bad! I meant simply giving them a link for macOS as well. But again that's probably not necessary.

[lcn2]

I meant simply giving them a link for macOS as well.

The problem would be to detect macOS and then to determine what link:

1. source code somewhere
2. HomeBrew command
3. Macports command

This is probably too much to do on behalf of the user for a system that lacks common command line tools?

[lcn2]

Something else just occurred to me: does anyone have any big endian machines to test everything under?

We don't. Do you think this is really needed?

[xexyl]

I meant simply giving them a link for macOS as well.

The problem would be to detect macOS and then to determine what link:

1. source code somewhere
2. HomeBrew command
3. Macports command

This is probably too much to do on behalf of the user for a system that lacks common command line tools?

True. I was thinking of the default location but as noted that's not even necessary. And in fact if they have the common command line tools they'll be able to compile the tools so it shouldn't be necessary anyway.

So forget that idea. Question is does the tar link for GNU need to be there?

[lcn2]

Question is does the tar link for GNU need to be there?

We don't think ti could hurt: or that could be removed if there is a problem.

[xexyl]

Question is does the tar link for GNU need to be there?

We don't think ti could hurt: or that could be removed if there is a problem.

I agree. It doesn't hurt at all to be there. I leave that up to you.

[xexyl]

Something else just occurred to me: does anyone have any big endian machines to test everything under?

We don't. Do you think this is really needed?

I don't: I only mentioned it because you had said you wanted to test as many systems as possible so I figured maybe bring up endianness as well.

[lcn2]

Something else just occurred to me: does anyone have any big endian machines to test everything under?

We don't. Do you think this is really needed?

I don't: I only mentioned it because you had said you wanted to test as many systems as possible so I figured maybe bring up endianness as well.

Fair point: however we don't have access to a big endian system.

[xexyl]

Something else just occurred to me: does anyone have any big endian machines to test everything under?

We don't. Do you think this is really needed?

I don't: I only mentioned it because you had said you wanted to test as many systems as possible so I figured maybe bring up endianness as well.

Fair point: however we don't have access to a big endian system.

That's fine. Besides you say in the guidelines that entries should use ASCII. Well maybe not should but you do mention it specifically iirc. That's me paraphrasing something that I've not read in a long time now of course but you know what I mean.

[xexyl]

Something else just occurred to me: does anyone have any big endian machines to test everything under?

We don't. Do you think this is really needed?

I don't: I only mentioned it because you had said you wanted to test as many systems as possible so I figured maybe bring up endianness as well.

Fair point: however we don't have access to a big endian system.

That's fine. Besides you say in the guidelines that entries should use ASCII. Well maybe not should but you do mention it specifically iirc. That's me paraphrasing something that I've not read in a long time now of course but you know what I mean.

Of course... I was thinking of versus EBCDIC but somehow in my head it was little endian versus big endian. Anyway I guess unless we can find someone who has a big endian machine we don't have to do anything about it but maybe we wouldn't need to anyway: not sure.

[xexyl]

The first one is change 1st, 2nd etc. to words. Similarly '2 args' should be 'two args' and so on ...

This item has been added to the TODO list above.

I can do the that one fairly easily but I don't want to stomp on any of your pending changes so I'm waiting on that. Hoping the pull request that I did today does not but as I said I can redo the JSON renaming again.

This has been done in commit f8fa3fd though I did point out that if it conflicts with your pending changes to json.c then reject the commit and I'll fix it again after you've made those changes. I don't mind either way.

[lcn2]

Updated, improved and expended on the checklist at the top.

[xexyl]

Updated, improved and expended on the checklist at the top.

Sounds good.

[xexyl]

Another todo: remove --report all from run_bison.sh invocation and remove the reference to jparse.output in make prep_clobber.

I've added this with commit 87ed3690cd97d5199a6df937a97d1453cf05a9e0 as it'll be useful for development.

[lcn2]

Another todo: remove --report all from run_bison.sh invocation and remove the reference to jparse.output in make prep_clobber.

Added to the above TODO list.

I've added this with commit 87ed369 as it'll be useful for development.

Should those 2 new TODO items be checked completed then?

[xexyl]

Another todo: remove --report all from run_bison.sh invocation and remove the reference to jparse.output in make prep_clobber.

Added to the above TODO list.

I've added this with commit 87ed369 as it'll be useful for development.

Should those 2 new TODO items be checked completed then?

No. I meant that I added the option in that commit and later on when it's written it can be removed.

[lcn2]

Another todo: remove --report all from run_bison.sh invocation and remove the reference to jparse.output in make prep_clobber.

Added to the above TODO list.

I've added this with commit 87ed369 as it'll be useful for development.

Should those 2 new TODO items be checked completed then?

No. I meant that I added the option in that commit and later on when it's written it can be removed.

So the 2 new TODO items should be removed now OR they should be checked as completed OR ???

[xexyl]

Another todo: remove --report all from run_bison.sh invocation and remove the reference to jparse.output in make prep_clobber.

Added to the above TODO list.

I've added this with commit 87ed369 as it'll be useful for development.

Should those 2 new TODO items be checked completed then?

No. I meant that I added the option in that commit and later on when it's written it can be removed.

So the 2 new TODO items should be removed now OR they should be checked as completed OR ???

Sorry for not making it clearer. They should be kept for now but once the parser is complete then we can look at doing it. So it should be on the todo list but for later. It's for after the json parser being done. I hope that helps.

EDIT: I will be leaving shortly and it's possible I won't be replying until tomorrow. I might but it's not certain. Good day!

[xexyl]

Thought of another thing that should be added to the list if it’s not already there:

Review the warnings that are disabled and make sure that they are truly needed and don’t mask any problems.

I can think of two that are needed for sure:

• CentOS requires -lm to link in the maths library for some of the functions (json specific I think). But under macOS this causes another warning that not all command line options are used. This can be ignored entirely. We could add it to the individual tools but this complicates it and it means we have to disable it for each new binary that links In json code.
• macOS has the strange idea that one is using unsafe paths even though we don’t use those paths so this can also (and is and should be) ignored.

However today I made a change to the Makefile to disable a warning for the jparse tool (I have a better way to go about it so it won’t be committed) but this is what made me think of this: some warnings are valid. The one in question was to not warn about unused parameters. But that’s actually a good warning to have for example.

Just something that might be a good idea to do.

[xexyl]

Another TODO item: check if older versions of flex and/or bison will work. I have different versions under CentOS and Fedora (though perhaps the upgrade I'm about to do will change that - I hope not) and both are under the required version (well for CentOS it's only bison that's not recent enough).

[lcn2]

CentOS requires -lm to link in the maths library for some of the functions (json specific I think). But under macOS this causes another warning that not all command line options are used.

We do not see those "command line options are used" warning under macOS.

This can be ignored entirely.

How? Not sure what you mean by "this" or "ignored" or even "entirely" in that context.

We could add it to the individual tools but this complicates it and it means we have to disable it for each new binary that links In json code.

That is a better approach. Please make the Makefile changes as you seem to see where the -lm is needed. In adding, we recommend the individual -lm be added before the '-o $@`.

[lcn2]

macOS has the strange idea that one is using unsafe paths even though we don’t use those paths so this can also (and is and should be) ignored.

We do not understand this point. Please explain more.

What would be the:

• [ ] something to do

line that should be added to the above TODO list?

[xexyl]

CentOS requires -lm to link in the maths library for some of the functions (json specific I think). But under macOS this causes another warning that not all command line options are used.

We do not see those "command line options are used" warning under macOS.

That's because I disabled the warning.

This can be ignored entirely.

How? Not sure what you mean by "this" or "ignored" or even "entirely" in that context.

Well is it a problem really if we have all binaries refer to LDFLAGS even if the maths library isn't used? That's all I am saying. macOS gave a warning that for some (those not linking in json) that the command line option is not used.

We could add it to the individual tools but this complicates it and it means we have to disable it for each new binary that links In json code.

That is a better approach. Please make the Makefile changes as you seem to see where the -lm is needed. In adding, we recommend the individual -lm be added before the '-o $@`.

It's in LDFLAGS - do you really want to add it to all the json ones and not the others? I mean it can be done but every time we add a new json util this has to be done when we can just have LDFLAGS at the end of CFLAGS. I can make the change if you like though it'll be tomorrow at the earliest.

[xexyl]

macOS has the strange idea that one is using unsafe paths even though we don’t use those paths so this can also (and is and should be) ignored.

We do not understand this point. Please explain more.

What would be the:

• [ ] something to do

line that should be added to the above TODO list?

Sorry for not being clear. The warning in particular that is disabled (in fact you did before I started working on the repo): -Wno-poison-system-directories.

As for what to include in the list perhaps something as simple as:

Verify that warnings that are disabled really should be disabled and that they're needed.

or maybe simpler still:

Make sure that we haven't disabled any warnings that matter.

or something like that.

The point is some warnings like the poison system directory one really are not a problem but maybe there are others (or will be others?) that are a problem. I don't actually know.

Is that more clear?

[lcn2]

LDFLAGS

The LDFLAGS should be reserved for linker flags are are required for ALL binaries.

If -lm if needed a a few cases, than perhaps that should be added to those specific binaries.

[lcn2]

macOS has the strange idea that one is using unsafe paths even though we don’t use those paths so this can also (and is and should be) ignored.

We do not understand this point. Please explain more. What would be the:

• [ ] something to do

line that should be added to the above TODO list?

Sorry for not being clear. The warning in particular that is disabled (in fact you did before I started working on the repo): -Wno-poison-system-directories.

As for what to include in the list perhaps something as simple as:

Verify that warnings that are disabled really should be disabled and that they're needed.

or maybe simpler still:

Make sure that we haven't disabled any warnings that matter.

or something like that.

The point is some warnings like the poison system directory one really are not a problem but maybe there are others (or will be others?) that are a problem. I don't actually know.

Is that more clear?

Added to the top list, thanks.

[xexyl]

LDFLAGS

The LDFLAGS should be reserved for linker flags are are required for ALL binaries.

If -lm if needed a a few cases, than perhaps that should be added to those specific binaries.

To be fair it's a matter of making it simpler and not so - the word is not coming to my mind. It also made it easier to test in that if a new tool uses that object file it's not left off by mistake. For example it's not actually needed under macOS (where I mostly work) but under both CentOS and Fedora it is needed.

So it's a matter of making it easier to make sure that it all compiles on all systems at all times. In the case that it's not needed there's no harm done because it won't be linked in.

I guess we could have a JSON_LDFLAGS or something like that that would be added to the json specific object files and binaries (and those that link it in)?

EDIT: Simpler and not so tedious and error prone would be how I'd put it. Error prone in that there's a risk that it fails compilation under systems I'm not working on (and nobody else either) and tedious in that it will take time to figure out which ones need it and which ones do not. If you want me to do that though I will.

[xexyl]

macOS has the strange idea that one is using unsafe paths even though we don’t use those paths so this can also (and is and should be) ignored.

We do not understand this point. Please explain more. What would be the:

• [ ] something to do

line that should be added to the above TODO list?

Sorry for not being clear. The warning in particular that is disabled (in fact you did before I started working on the repo): -Wno-poison-system-directories. As for what to include in the list perhaps something as simple as:

Verify that warnings that are disabled really should be disabled and that they're needed.

or maybe simpler still:

Make sure that we haven't disabled any warnings that matter.

or something like that. The point is some warnings like the poison system directory one really are not a problem but maybe there are others (or will be others?) that are a problem. I don't actually know. Is that more clear?

Added to the top list, thanks.

Welcome.

[xexyl]

As for:

Modify txzchk to be able to read a file as it it were the output of some tar -t... command
Build a directory of both good and bad *.txt files (instead of good and bad tarballs` as test cases.

txzchk already has this. It was an early feature. The only thing that has to be done is write the script and come up with some bad and good files. I can help with the files. I already have some. Want me to add a test_txzchk (or maybe test_txz) directory with good and bad subdirectories with some files?

BTW: the man page has an example of how this works:

       Run the program on the file entry.test-1.1644094311.txt, specifying that it's a text file (which means -T and -E have to be used):

               ./txzchk -T -E txt entry.test-1.1644094311.txt

-E changes the extension (for fnamchk) and -T specifies that it's a text file rather than a tarball.

[xexyl]

As for portability I just did a quick search through some man pages of tar on different systems and it appears that:

• NetBSD has -J with XZ meaning
• OpenBSD does not have -J at all
• FreeBSD does have -J with XZ meaning
• IRIX does not have -J with XZ meaning (or at all I think)

I believe the ones that do not do have bzip2 however. I'm not sure of SunOS / Solaris. Neither am I sure of they all have the v7 format.

Just something that might or might not be relevant. I'm curious if NetBSD and FreeBSD have the same format listing as macOS or not. I've not checked the source code but maybe that wouldn't be the best way to go about it anyway. Either way I'm doing something else now and I have a doctor appointment later today.

Hope you have a great day! We can discuss the dbg documentation though like I said elsewhere I doubt I'll be doing any more changes than the one commit I did a bit ago (removed invalid bad test cases with txzchk).

[xexyl]

I believe the ones that do not do have bzip2 however. I'm not sure of SunOS / Solaris. Neither am I sure of they all have the v7 format.

FreeBSD apparently has:

--numeric-owner
         This is equivalent to --uname "" --gname "".  On extract, it
         causes user and group names in the archive to be ignored in favor
         of the numeric user and group ids.  On create, it causes user and
         group names to not be stored in the archive.

for example. macOS also has this option as well as linux. Not sure about the others. But neither am I sure if the v7 format is only for that? If it is only that maybe it should be updated? I don't know. Then again it might be the man pages of those systems don't show that format. Not sure.

UPDATE: Looking at the man page for macOS it appears it might be an undocumented format:

    --format format
             (c, r, u mode only) Use the specified format for the created archive.  Supported formats include "cpio", "pax", "shar", and
             "ustar".  Other formats may also be supported; see libarchive-formats(5) for more information about currently-supported
             formats.  In r and u modes, when extending an existing archive, the format specified here must be compatible with the
             format of the existing archive on disk.

No reference to v7. Maybe the man pages for the BSDs are also that way? Not sure of the -J option but I suspect not somehow especially for IRIX.

Ah. For macOS it's under the man page referred to. Same with FreeBSD. Probably the other BSDs as well. Anyway going afk now. Good day!